2024-03-29T03:56:35.224 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-29T03:56:35.225 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076 --description fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} --owner scheduled_mchangir@teuthology --name mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi --verbose -- /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076/orig.config.yaml 2024-03-29T03:56:35.274 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T03:56:35.364 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076 branch: wip-mchangir-testing1-main-20240328-072412 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} email: mchangir@redhat.com first_in_suite: false job_id: '7629076' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing1-main-20240328-072412 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing1-main-20240328-072412 sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 owner: scheduled_mchangir@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8207 sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-mchangir-testing1-main-20240328-072412 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIZEzUaPO6wk7oZ1TFA+ugiuBrfnoH0JnaSZXeUIPE1MO0Hym+6XkM68t6vlyVE4ughsiaFekDbZCWB3/gc3H/Y= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEWPhAtYviuzhJO7VkwnqL3RDKbYNrsCin0sYC7fKMpz9KQzEDIwmoNQsE1m1jOSCNOscybOnMO0aANEZJoPBJs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_uninlining teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-29_02:01:30 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-29T03:56:35.364 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa; will attempt to use it 2024-03-29T03:56:35.365 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks 2024-03-29T03:56:35.365 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-29T03:56:35.366 INFO:teuthology.task.internal:Checking packages... 2024-03-29T03:56:35.394 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '69e2cd73c52284dfd5096c66b60c6e6b26a3c447' 2024-03-29T03:56:35.394 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T03:56:35.394 INFO:teuthology.packaging:ref: None 2024-03-29T03:56:35.394 INFO:teuthology.packaging:tag: None 2024-03-29T03:56:35.394 INFO:teuthology.packaging:branch: wip-mchangir-testing1-main-20240328-072412 2024-03-29T03:56:35.394 INFO:teuthology.packaging:sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T03:56:35.395 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing1-main-20240328-072412 2024-03-29T03:56:35.545 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2520.g69e2cd73 2024-03-29T03:56:35.547 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-29T03:56:35.553 INFO:teuthology.task.internal:no buildpackages task found 2024-03-29T03:56:35.553 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-29T03:56:35.570 INFO:teuthology.task.internal:Saving configuration 2024-03-29T03:56:35.589 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-29T03:56:35.595 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T03:56:35.620 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:51:28.156004', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIZEzUaPO6wk7oZ1TFA+ugiuBrfnoH0JnaSZXeUIPE1MO0Hym+6XkM68t6vlyVE4ughsiaFekDbZCWB3/gc3H/Y='} 2024-03-29T03:56:35.644 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:51:28.157853', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEWPhAtYviuzhJO7VkwnqL3RDKbYNrsCin0sYC7fKMpz9KQzEDIwmoNQsE1m1jOSCNOscybOnMO0aANEZJoPBJs='} 2024-03-29T03:56:35.645 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-29T03:56:35.651 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-29T03:56:35.651 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-29T03:56:35.651 INFO:teuthology.run_tasks:Running task console_log... 2024-03-29T03:56:35.730 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8466175f70>, signals=[15]) 2024-03-29T03:56:35.730 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-29T03:56:35.736 INFO:teuthology.task.internal:Opening connections... 2024-03-29T03:56:35.736 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T03:56:35.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T03:56:35.831 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T03:56:35.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T03:56:35.926 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-29T03:56:35.933 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-03-29T03:56:35.951 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-03-29T03:56:35.953 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-03-29T03:56:36.007 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-03-29T03:56:36.008 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-03-29T03:56:36.008 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-03-29T03:56:36.008 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-03-29T03:56:36.008 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-03-29T03:56:36.008 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-03-29T03:56:36.008 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-29T03:56:36.008 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-03-29T03:56:36.008 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-03-29T03:56:36.008 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-29T03:56:36.008 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-03-29T03:56:36.008 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-29T03:56:36.009 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-29T03:56:36.009 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-29T03:56:36.009 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-03-29T03:56:36.041 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2024-03-29T03:56:36.059 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2024-03-29T03:56:36.060 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2024-03-29T03:56:36.116 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2024-03-29T03:56:36.116 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9" 2024-03-29T03:56:36.116 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2024-03-29T03:56:36.116 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2024-03-29T03:56:36.116 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9" 2024-03-29T03:56:36.116 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9" 2024-03-29T03:56:36.116 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-29T03:56:36.117 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2024-03-29T03:56:36.117 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon" 2024-03-29T03:56:36.117 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-29T03:56:36.117 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2024-03-29T03:56:36.117 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-29T03:56:36.117 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-29T03:56:36.117 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-29T03:56:36.117 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2024-03-29T03:56:36.141 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-29T03:56:36.150 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-29T03:56:36.156 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-29T03:56:36.157 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-03-29T03:56:36.160 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2024-03-29T03:56:36.176 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-29T03:56:36.182 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-29T03:56:36.182 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-03-29T03:56:36.217 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2024-03-29T03:56:36.254 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-29T03:56:36.308 INFO:teuthology.run_tasks:Running task kernel... 2024-03-29T03:56:36.324 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-29T03:56:36.325 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-03-29T03:56:36.325 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-03-29T03:56:36.325 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T03:56:36.326 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T03:56:36.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T03:56:36.342 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-03-29T03:56:36.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T03:56:36.343 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-03-29T03:56:36.400 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-361.el9.x86_64 2024-03-29T03:56:36.400 INFO:teuthology.task.kernel:Running kernel on smithi097: 5.14.0-361.el9.x86_64 2024-03-29T03:56:36.400 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum install -y kernel 2024-03-29T03:56:36.402 INFO:teuthology.orchestra.run.smithi154.stdout:5.14.0-361.el9.x86_64 2024-03-29T03:56:36.402 INFO:teuthology.task.kernel:Running kernel on smithi154: 5.14.0-361.el9.x86_64 2024-03-29T03:56:36.402 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum install -y kernel 2024-03-29T03:56:38.548 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-29T03:56:38.550 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-29T03:56:39.985 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - BaseOS 6.2 MB/s | 2.0 MB 00:00 2024-03-29T03:56:39.991 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.0 MB 00:00 2024-03-29T03:56:40.748 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-03-29T03:56:41.292 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.0 MB 00:00 2024-03-29T03:56:42.410 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 32 MB/s | 19 MB 00:00 2024-03-29T03:56:43.654 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-03-29T03:56:46.627 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.1 MB 00:00 2024-03-29T03:56:48.243 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-03-29T03:56:50.026 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.1 MB 00:02 2024-03-29T03:56:50.446 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-03-29T03:56:51.360 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 16 kB 00:00 2024-03-29T03:56:52.998 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-29T03:56:55.302 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-03-29T03:56:57.534 INFO:teuthology.orchestra.run.smithi097.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-29T03:56:57.534 INFO:teuthology.orchestra.run.smithi097.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-29T03:56:57.626 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-29T03:56:57.637 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-29T03:56:57.637 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-29T03:56:57.637 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-29T03:56:57.637 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-29T03:56:57.637 INFO:teuthology.orchestra.run.smithi097.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-03-29T03:56:57.637 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-29T03:56:57.637 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-03-29T03:56:57.638 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-03-29T03:56:57.638 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-03-29T03:56:57.638 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T03:56:57.638 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-29T03:56:57.638 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-03-29T03:56:57.638 INFO:teuthology.orchestra.run.smithi097.stdout:Install 4 Packages 2024-03-29T03:56:57.638 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T03:56:57.638 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 76 M 2024-03-29T03:56:57.638 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 123 M 2024-03-29T03:56:57.638 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-29T03:56:57.811 INFO:teuthology.orchestra.run.smithi097.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-03-29T03:56:58.236 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-29T03:56:58.337 INFO:teuthology.orchestra.run.smithi097.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-03-29T03:56:58.770 INFO:teuthology.orchestra.run.smithi097.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 30 MB/s | 34 MB 00:01 2024-03-29T03:56:59.212 INFO:teuthology.orchestra.run.smithi097.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 20 MB/s | 28 MB 00:01 2024-03-29T03:56:59.215 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-29T03:56:59.215 INFO:teuthology.orchestra.run.smithi097.stdout:Total 48 MB/s | 76 MB 00:01 2024-03-29T03:56:59.215 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-29T03:56:59.566 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-29T03:56:59.566 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-29T03:57:00.434 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-29T03:57:00.434 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-29T03:57:00.535 INFO:teuthology.orchestra.run.smithi154.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-29T03:57:00.535 INFO:teuthology.orchestra.run.smithi154.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-29T03:57:00.624 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-03-29T03:57:00.628 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-03-29T03:57:00.628 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-03-29T03:57:00.629 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-03-29T03:57:00.629 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-03-29T03:57:00.629 INFO:teuthology.orchestra.run.smithi154.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-03-29T03:57:00.629 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-03-29T03:57:00.629 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-03-29T03:57:00.629 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-03-29T03:57:00.630 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-03-29T03:57:00.630 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T03:57:00.630 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-03-29T03:57:00.630 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-03-29T03:57:00.630 INFO:teuthology.orchestra.run.smithi154.stdout:Install 4 Packages 2024-03-29T03:57:00.630 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T03:57:00.631 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 76 M 2024-03-29T03:57:00.632 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 123 M 2024-03-29T03:57:00.632 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-03-29T03:57:00.771 INFO:teuthology.orchestra.run.smithi154.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-29T03:57:01.330 INFO:teuthology.orchestra.run.smithi154.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-03-29T03:57:01.855 INFO:teuthology.orchestra.run.smithi154.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 27 MB/s | 34 MB 00:01 2024-03-29T03:57:02.197 INFO:teuthology.orchestra.run.smithi154.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 19 MB/s | 28 MB 00:01 2024-03-29T03:57:02.199 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-03-29T03:57:02.200 INFO:teuthology.orchestra.run.smithi154.stdout:Total 49 MB/s | 76 MB 00:01 2024-03-29T03:57:02.200 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-03-29T03:57:02.578 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-03-29T03:57:02.579 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-03-29T03:57:02.746 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-29T03:57:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-03-29T03:57:03.427 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-03-29T03:57:03.427 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-03-29T03:57:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:57:05.782 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-03-29T03:57:06.201 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:57:06.340 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T03:57:06.372 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-03-29T03:57:06.456 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:57:09.194 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:57:09.319 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T03:57:11.088 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T03:57:11.189 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:57:14.032 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T03:57:14.301 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:57:15.971 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:57:19.052 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:57:45.441 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:57:45.441 INFO:teuthology.orchestra.run.smithi097.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T03:57:45.441 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T03:57:45.483 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:57:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:57:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-03-29T03:57:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:57:45.905 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T03:57:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:57:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T03:57:46.315 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-29T03:57:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-5.14.0-434.el9.x86_64 2024-03-29T03:57:46.315 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-03-29T03:57:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-03-29T03:57:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-03-29T03:57:46.316 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T03:57:46.316 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-29T03:57:46.638 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T03:57:46.737 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T03:57:46.737 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T03:57:46.737 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-03-29T03:57:46.737 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-29T03:57:46.738 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-29T03:57:46.738 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-29T03:57:46.738 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi097.front.sepia.ceph.com, path=None, version=distro) 2024-03-29T03:57:46.738 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T03:57:46.819 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T03:57:46.819 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T03:57:46.819 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -qi grub2-tools 2024-03-29T03:57:46.853 INFO:teuthology.orchestra.run.smithi097.stdout:Name : grub2-tools 2024-03-29T03:57:46.853 INFO:teuthology.orchestra.run.smithi097.stdout:Epoch : 1 2024-03-29T03:57:46.853 INFO:teuthology.orchestra.run.smithi097.stdout:Version : 2.06 2024-03-29T03:57:46.853 INFO:teuthology.orchestra.run.smithi097.stdout:Release : 68.el9 2024-03-29T03:57:46.854 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: x86_64 2024-03-29T03:57:46.854 INFO:teuthology.orchestra.run.smithi097.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-29T03:57:46.854 INFO:teuthology.orchestra.run.smithi097.stdout:Group : Unspecified 2024-03-29T03:57:46.854 INFO:teuthology.orchestra.run.smithi097.stdout:Size : 8264254 2024-03-29T03:57:46.854 INFO:teuthology.orchestra.run.smithi097.stdout:License : GPLv3+ 2024-03-29T03:57:46.854 INFO:teuthology.orchestra.run.smithi097.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-29T03:57:46.854 INFO:teuthology.orchestra.run.smithi097.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-29T03:57:46.854 INFO:teuthology.orchestra.run.smithi097.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-29T03:57:46.854 INFO:teuthology.orchestra.run.smithi097.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-29T03:57:46.854 INFO:teuthology.orchestra.run.smithi097.stdout:Packager : builder@centos.org 2024-03-29T03:57:46.854 INFO:teuthology.orchestra.run.smithi097.stdout:Vendor : CentOS 2024-03-29T03:57:46.854 INFO:teuthology.orchestra.run.smithi097.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-29T03:57:46.855 INFO:teuthology.orchestra.run.smithi097.stdout:Summary : Support tools for GRUB. 2024-03-29T03:57:46.855 INFO:teuthology.orchestra.run.smithi097.stdout:Description : 2024-03-29T03:57:46.855 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T03:57:46.855 INFO:teuthology.orchestra.run.smithi097.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-29T03:57:46.855 INFO:teuthology.orchestra.run.smithi097.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-29T03:57:46.855 INFO:teuthology.orchestra.run.smithi097.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-29T03:57:46.855 INFO:teuthology.orchestra.run.smithi097.stdout:hardware devices. 2024-03-29T03:57:46.855 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T03:57:46.855 INFO:teuthology.orchestra.run.smithi097.stdout:This subpackage provides tools for support of all platforms. 2024-03-29T03:57:46.856 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-29T03:57:46.856 INFO:teuthology.task.kernel:Updating grub on smithi097 to boot 5.14.0-434.el9.x86_64 2024-03-29T03:57:46.856 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-29T03:57:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-03-29T03:57:48.460 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:57:48.460 INFO:teuthology.orchestra.run.smithi154.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T03:57:48.460 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T03:57:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:57:48.847 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:57:48.847 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-03-29T03:57:48.847 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:57:48.847 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T03:57:48.850 INFO:teuthology.orchestra.run.smithi097.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-29T03:57:48.889 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-03-29T03:57:48.892 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-03-29T03:57:48.907 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.Ip6L6AIb0b 2024-03-29T03:57:48.907 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Ip6L6AIb0b 2024-03-29T03:57:48.975 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.Ip6L6AIb0b 2024-03-29T03:57:49.113 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.Ip6L6AIb0b is 6KB 2024-03-29T03:57:49.161 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.Ip6L6AIb0b 2024-03-29T03:57:49.175 DEBUG:teuthology.orchestra.run.smithi097:> sudo /bin/ls /boot/loader/entries || true 2024-03-29T03:57:49.240 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-29T03:57:49.240 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-29T03:57:49.240 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-29T03:57:49.240 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-03-29T03:57:49.241 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-03-29T03:57:49.261 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:57:49.261 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T03:57:49.261 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-03-29T03:57:49.261 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-5.14.0-434.el9.x86_64 2024-03-29T03:57:49.261 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-03-29T03:57:49.262 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-03-29T03:57:49.262 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-03-29T03:57:49.262 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T03:57:49.262 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-03-29T03:57:49.396 DEBUG:teuthology.orchestra.run.smithi097:> sudo shutdown -r now 2024-03-29T03:57:49.590 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T03:57:49.694 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T03:57:49.694 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T03:57:49.695 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-03-29T03:57:49.695 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-29T03:57:49.695 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-29T03:57:49.695 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-03-29T03:57:49.695 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi154.front.sepia.ceph.com, path=None, version=distro) 2024-03-29T03:57:49.695 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T03:57:49.776 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T03:57:49.776 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T03:57:49.776 DEBUG:teuthology.orchestra.run.smithi154:> sudo rpm -qi grub2-tools 2024-03-29T03:57:49.810 INFO:teuthology.orchestra.run.smithi154.stdout:Name : grub2-tools 2024-03-29T03:57:49.810 INFO:teuthology.orchestra.run.smithi154.stdout:Epoch : 1 2024-03-29T03:57:49.810 INFO:teuthology.orchestra.run.smithi154.stdout:Version : 2.06 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:Release : 68.el9 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: x86_64 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:Group : Unspecified 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:Size : 8264254 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:License : GPLv3+ 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:Packager : builder@centos.org 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:Vendor : CentOS 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-29T03:57:49.811 INFO:teuthology.orchestra.run.smithi154.stdout:Summary : Support tools for GRUB. 2024-03-29T03:57:49.812 INFO:teuthology.orchestra.run.smithi154.stdout:Description : 2024-03-29T03:57:49.812 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T03:57:49.812 INFO:teuthology.orchestra.run.smithi154.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-29T03:57:49.812 INFO:teuthology.orchestra.run.smithi154.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-29T03:57:49.812 INFO:teuthology.orchestra.run.smithi154.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-29T03:57:49.812 INFO:teuthology.orchestra.run.smithi154.stdout:hardware devices. 2024-03-29T03:57:49.812 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T03:57:49.812 INFO:teuthology.orchestra.run.smithi154.stdout:This subpackage provides tools for support of all platforms. 2024-03-29T03:57:49.813 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-29T03:57:49.813 INFO:teuthology.task.kernel:Updating grub on smithi154 to boot 5.14.0-434.el9.x86_64 2024-03-29T03:57:49.813 DEBUG:teuthology.orchestra.run.smithi154:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-29T03:57:50.452 INFO:teuthology.orchestra.run.smithi154.stderr:Generating grub configuration file ... 2024-03-29T03:57:51.792 INFO:teuthology.orchestra.run.smithi154.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-29T03:57:51.829 INFO:teuthology.orchestra.run.smithi154.stderr:done 2024-03-29T03:57:51.832 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-03-29T03:57:51.864 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.UvjVmpLJC4 2024-03-29T03:57:51.865 DEBUG:teuthology.orchestra.run.smithi154:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UvjVmpLJC4 2024-03-29T03:57:51.933 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.UvjVmpLJC4 2024-03-29T03:57:52.075 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.UvjVmpLJC4 is 6KB 2024-03-29T03:57:52.124 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.UvjVmpLJC4 2024-03-29T03:57:52.138 DEBUG:teuthology.orchestra.run.smithi154:> sudo /bin/ls /boot/loader/entries || true 2024-03-29T03:57:52.202 INFO:teuthology.orchestra.run.smithi154.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-29T03:57:52.203 INFO:teuthology.orchestra.run.smithi154.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-29T03:57:52.203 INFO:teuthology.orchestra.run.smithi154.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-29T03:57:52.203 INFO:teuthology.orchestra.run.smithi154.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-03-29T03:57:52.204 DEBUG:teuthology.orchestra.run.smithi154:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-03-29T03:57:52.376 DEBUG:teuthology.orchestra.run.smithi154:> sudo shutdown -r now 2024-03-29T03:58:19.422 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-29T03:58:19.423 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-29T03:58:19.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T03:58:22.382 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-29T03:58:22.382 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-03-29T03:58:22.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T03:58:57.055 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2024-03-29T03:59:06.065 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-03-29T03:59:06.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T03:59:19.440 DEBUG:teuthology.orchestra.remote:timed out 2024-03-29T03:59:28.446 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-29T03:59:28.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T03:59:29.847 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-03-29T03:59:30.387 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-29T03:59:30.388 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-434.el9.x86_64"... 2024-03-29T03:59:30.388 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-03-29T03:59:30.402 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-434.el9.x86_64 2024-03-29T03:59:30.402 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-03-29T03:59:30.402 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T03:59:30.403 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-29T03:59:31.403 DEBUG:teuthology.parallel:result is None 2024-03-29T04:00:06.069 DEBUG:teuthology.orchestra.remote:timed out 2024-03-29T04:00:18.070 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-03-29T04:00:18.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:00:18.746 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-03-29T04:00:19.106 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-03-29T04:00:19.106 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-434.el9.x86_64"... 2024-03-29T04:00:19.107 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-03-29T04:00:19.122 INFO:teuthology.orchestra.run.smithi154.stdout:5.14.0-434.el9.x86_64 2024-03-29T04:00:19.122 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-03-29T04:00:19.122 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T04:00:19.122 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-29T04:00:20.123 DEBUG:teuthology.parallel:result is None 2024-03-29T04:00:20.124 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-29T04:00:20.131 INFO:teuthology.task.internal:Creating test directory... 2024-03-29T04:00:20.131 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T04:00:20.135 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T04:00:20.152 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-29T04:00:20.159 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-29T04:00:20.219 INFO:teuthology.task.internal:Creating archive directory... 2024-03-29T04:00:20.220 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T04:00:20.222 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T04:00:20.474 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-29T04:00:20.481 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-29T04:00:20.481 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T04:00:20.484 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T04:00:20.575 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:00:20.592 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:00:20.613 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:00:20.624 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:00:20.626 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-29T04:00:20.634 INFO:teuthology.task.internal:Configuring sudo... 2024-03-29T04:00:20.634 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T04:00:20.636 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T04:00:20.666 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-29T04:00:20.675 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-29T04:00:20.675 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T04:00:20.706 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T04:00:20.722 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:00:20.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:00:20.855 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:00:20.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:00:20.980 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:00:20.980 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T04:00:21.048 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:00:21.094 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:00:21.159 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:00:21.201 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:00:21.245 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:00:21.246 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T04:00:21.314 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-03-29T04:00:21.316 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2024-03-29T04:00:21.369 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:00:21.399 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:00:21.874 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-29T04:00:21.882 INFO:teuthology.task.internal:Starting timer... 2024-03-29T04:00:21.883 INFO:teuthology.run_tasks:Running task pcp... 2024-03-29T04:00:21.896 INFO:teuthology.run_tasks:Running task selinux... 2024-03-29T04:00:21.903 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-03-29T04:00:21.904 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-03-29T04:00:21.969 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-03-29T04:00:21.972 DEBUG:teuthology.orchestra.run.smithi154:> sudo service auditd rotate 2024-03-29T04:00:22.037 INFO:teuthology.orchestra.run.smithi154.stdout:Rotating logs: 2024-03-29T04:00:22.039 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-29T04:00:22.039 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-03-29T04:00:22.061 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-03-29T04:00:22.061 DEBUG:teuthology.orchestra.run.smithi154:> /usr/sbin/getenforce 2024-03-29T04:00:22.087 INFO:teuthology.orchestra.run.smithi154.stdout:Permissive 2024-03-29T04:00:22.088 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi154.front.sepia.ceph.com': 'permissive'} 2024-03-29T04:00:22.088 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:00:22.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:00:22.131 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:00:22.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:00:22.159 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-29T04:00:22.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-03-29T04:00:22.204 DEBUG:teuthology.orchestra.run.smithi154:> sudo /usr/sbin/setenforce permissive 2024-03-29T04:00:22.239 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-29T04:00:22.249 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-29T04:00:22.267 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-29T04:00:22.268 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi154.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-29T04:04:37.753 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com')] 2024-03-29T04:04:37.754 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-29T04:04:37.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:04:37.846 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-03-29T04:04:37.928 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-03-29T04:04:37.929 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-03-29T04:04:37.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:04:38.018 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-03-29T04:04:38.097 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-03-29T04:04:38.097 INFO:teuthology.run_tasks:Running task clock... 2024-03-29T04:04:38.109 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-29T04:04:38.109 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T04:04:38.109 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:04:38.112 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T04:04:38.112 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:04:38.140 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-29T04:04:38.157 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-29T04:04:38.168 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-29T04:04:38.185 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-29T04:04:38.186 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-03-29T04:04:38.200 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-03-29T04:04:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: ntpd: command not found 2024-03-29T04:04:38.217 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-29T04:04:38.229 INFO:teuthology.orchestra.run.smithi154.stdout:506 Cannot talk to daemon 2024-03-29T04:04:38.233 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-29T04:04:38.245 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-29T04:04:38.261 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-29T04:04:38.291 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:04:38.294 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:04:38.294 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-03-29T04:04:38.294 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:04:38.294 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:04:38.294 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:04:38.294 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:04:38.318 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:04:38.322 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:04:38.322 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-03-29T04:04:38.322 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:04:38.322 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:04:38.323 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:04:38.323 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:04:38.323 INFO:teuthology.run_tasks:Running task install... 2024-03-29T04:04:38.331 DEBUG:teuthology.task.install:project ceph 2024-03-29T04:04:38.331 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}} 2024-03-29T04:04:38.331 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'} 2024-03-29T04:04:38.331 INFO:teuthology.task.install:Using flavor: default 2024-03-29T04:04:38.337 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-29T04:04:38.337 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-29T04:04:38.338 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'tag': None, 'wait_for_package': False} 2024-03-29T04:04:38.338 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:04:38.340 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'tag': None, 'wait_for_package': False} 2024-03-29T04:04:38.341 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:04:38.564 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/ 2024-03-29T04:04:38.564 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2520.g69e2cd73 2024-03-29T04:04:38.578 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/ 2024-03-29T04:04:38.578 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2520.g69e2cd73 2024-03-29T04:04:38.734 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-29T04:04:38.735 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:04:38.735 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-29T04:04:38.745 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-29T04:04:38.745 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:04:38.745 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-29T04:04:38.763 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-29T04:04:38.763 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-29T04:04:38.774 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-29T04:04:38.774 DEBUG:teuthology.orchestra.run.smithi154:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-29T04:04:38.832 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-29T04:04:38.842 DEBUG:teuthology.orchestra.run.smithi154:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-29T04:04:38.913 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-29T04:04:38.922 DEBUG:teuthology.orchestra.run.smithi154:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-29T04:04:38.981 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-03-29T04:04:38.982 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-03-29T04:04:38.988 INFO:teuthology.orchestra.run.smithi154.stdout:check_obsoletes = 1 2024-03-29T04:04:38.990 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum clean all 2024-03-29T04:04:39.226 INFO:teuthology.orchestra.run.smithi097.stdout:74 files removed 2024-03-29T04:04:39.239 INFO:teuthology.orchestra.run.smithi154.stdout:74 files removed 2024-03-29T04:04:39.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-29T04:04:39.266 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-29T04:04:39.873 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 263 kB/s | 83 kB 00:00 2024-03-29T04:04:39.911 INFO:teuthology.orchestra.run.smithi154.stdout:ceph packages for x86_64 243 kB/s | 83 kB 00:00 2024-03-29T04:04:40.240 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-03-29T04:04:40.245 INFO:teuthology.orchestra.run.smithi154.stdout:ceph noarch packages 41 kB/s | 12 kB 00:00 2024-03-29T04:04:40.641 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-03-29T04:04:40.670 INFO:teuthology.orchestra.run.smithi154.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-03-29T04:04:41.073 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-29T04:04:41.369 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - AppStream 9.7 MB/s | 6.7 MB 00:00 2024-03-29T04:04:42.481 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-03-29T04:04:42.752 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.0 MB 00:00 2024-03-29T04:04:43.339 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-03-29T04:04:45.573 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-03-29T04:04:49.850 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 8.0 MB/s | 6.1 MB 00:00 2024-03-29T04:04:51.492 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-03-29T04:04:52.150 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 907 kB/s | 8.0 MB 00:09 2024-03-29T04:04:53.152 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-29T04:04:56.451 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-03-29T04:04:58.102 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-03-29T04:05:00.681 INFO:teuthology.orchestra.run.smithi154.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-29T04:05:00.682 INFO:teuthology.orchestra.run.smithi154.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-29T04:05:00.696 INFO:teuthology.orchestra.run.smithi154.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-29T04:05:00.697 INFO:teuthology.orchestra.run.smithi154.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-29T04:05:00.698 INFO:teuthology.orchestra.run.smithi154.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-29T04:05:00.699 INFO:teuthology.orchestra.run.smithi154.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-29T04:05:00.702 INFO:teuthology.orchestra.run.smithi154.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-29T04:05:00.705 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-29T04:05:00.706 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-29T04:05:00.788 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-03-29T04:05:00.810 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.1 MB 00:00 2024-03-29T04:05:00.814 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2024-03-29T04:05:00.814 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-03-29T04:05:00.814 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2024-03-29T04:05:00.814 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-03-29T04:05:00.814 INFO:teuthology.orchestra.run.smithi154.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-29T04:05:00.814 INFO:teuthology.orchestra.run.smithi154.stdout: ceph x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 6.6 k 2024-03-29T04:05:00.814 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 5.4 M 2024-03-29T04:05:00.814 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 890 k 2024-03-29T04:05:00.814 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 145 k 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 1.5 M 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 141 k 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 3.5 M 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 7.4 M 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 49 k 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 9.8 M 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 48 M 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 264 k 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 762 k 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 221 k 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 27 k 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 18 k 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-29T04:05:00.815 INFO:teuthology.orchestra.run.smithi154.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-29T04:05:00.816 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-29T04:05:00.816 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 32 k 2024-03-29T04:05:00.816 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 746 k 2024-03-29T04:05:00.816 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 127 k 2024-03-29T04:05:00.816 INFO:teuthology.orchestra.run.smithi154.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-29T04:05:00.816 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-29T04:05:00.816 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-29T04:05:00.816 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-03-29T04:05:00.816 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-03-29T04:05:00.816 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-03-29T04:05:00.816 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 162 k 2024-03-29T04:05:00.816 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 321 k 2024-03-29T04:05:00.817 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 298 k 2024-03-29T04:05:00.817 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 99 k 2024-03-29T04:05:00.817 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 85 k 2024-03-29T04:05:00.817 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.1 M 2024-03-29T04:05:00.817 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 170 k 2024-03-29T04:05:00.817 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-29T04:05:00.817 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrading: 2024-03-29T04:05:00.817 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-03-29T04:05:00.817 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-03-29T04:05:00.817 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-03-29T04:05:00.817 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-03-29T04:05:00.817 INFO:teuthology.orchestra.run.smithi154.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-29T04:05:00.817 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.4 M 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.1 M 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-29T04:05:00.818 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 21 M 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 34 k 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 2.4 M 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 249 k 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 4.7 M 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 17 M 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 16 k 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 25 k 2024-03-29T04:05:00.819 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 163 k 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-29T04:05:00.820 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 500 k 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 5.1 M 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-29T04:05:00.821 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 45 k 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 134 k 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-29T04:05:00.822 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-29T04:05:00.823 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-29T04:05:00.824 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-29T04:05:00.825 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-29T04:05:00.825 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-29T04:05:00.825 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-29T04:05:00.825 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-29T04:05:00.825 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-29T04:05:00.825 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-29T04:05:00.825 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-29T04:05:00.825 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-29T04:05:00.825 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-29T04:05:00.825 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-29T04:05:00.825 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-29T04:05:00.826 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-29T04:05:00.826 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-29T04:05:00.826 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-29T04:05:00.826 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-29T04:05:00.827 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-29T04:05:00.827 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-29T04:05:00.827 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-29T04:05:00.827 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-29T04:05:00.827 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-03-29T04:05:00.827 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-03-29T04:05:00.827 INFO:teuthology.orchestra.run.smithi154.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-29T04:05:00.827 INFO:teuthology.orchestra.run.smithi154.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-29T04:05:00.827 INFO:teuthology.orchestra.run.smithi154.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-29T04:05:00.827 INFO:teuthology.orchestra.run.smithi154.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-29T04:05:00.827 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-29T04:05:00.827 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-29T04:05:00.827 INFO:teuthology.orchestra.run.smithi154.stdout:Installing weak dependencies: 2024-03-29T04:05:00.828 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-03-29T04:05:00.828 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:05:00.828 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-03-29T04:05:00.828 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2024-03-29T04:05:00.828 INFO:teuthology.orchestra.run.smithi154.stdout:Install 150 Packages 2024-03-29T04:05:00.828 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrade 24 Packages 2024-03-29T04:05:00.828 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:05:00.829 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 226 M 2024-03-29T04:05:00.829 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-03-29T04:05:02.153 INFO:teuthology.orchestra.run.smithi154.stdout:(1/174): ceph-19.0.0-2520.g69e2cd73.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-03-29T04:05:02.269 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-03-29T04:05:03.819 INFO:teuthology.orchestra.run.smithi154.stdout:(2/174): ceph-fuse-19.0.0-2520.g69e2cd73.el9.x8 534 kB/s | 890 kB 00:01 2024-03-29T04:05:04.153 INFO:teuthology.orchestra.run.smithi154.stdout:(3/174): ceph-immutable-object-cache-19.0.0-252 435 kB/s | 145 kB 00:00 2024-03-29T04:05:04.620 INFO:teuthology.orchestra.run.smithi154.stdout:(4/174): ceph-common-19.0.0-2520.g69e2cd73.el9. 6.9 MB/s | 21 MB 00:02 2024-03-29T04:05:05.078 INFO:teuthology.orchestra.run.smithi154.stdout:(5/174): ceph-mgr-19.0.0-2520.g69e2cd73.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-03-29T04:05:05.595 INFO:teuthology.orchestra.run.smithi154.stdout:(6/174): ceph-base-19.0.0-2520.g69e2cd73.el9.x8 1.4 MB/s | 5.4 MB 00:03 2024-03-29T04:05:10.754 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 2.5 MB/s | 21 MB 00:08 2024-03-29T04:05:12.020 INFO:teuthology.orchestra.run.smithi154.stdout:(7/174): ceph-mon-19.0.0-2520.g69e2cd73.el9.x86 693 kB/s | 4.7 MB 00:06 2024-03-29T04:05:12.395 INFO:teuthology.orchestra.run.smithi154.stdout:(8/174): ceph-mds-19.0.0-2520.g69e2cd73.el9.x86 295 kB/s | 2.4 MB 00:08 2024-03-29T04:05:12.512 INFO:teuthology.orchestra.run.smithi154.stdout:(9/174): ceph-selinux-19.0.0-2520.g69e2cd73.el9 216 kB/s | 25 kB 00:00 2024-03-29T04:05:13.420 INFO:teuthology.orchestra.run.smithi154.stdout:(10/174): ceph-radosgw-19.0.0-2520.g69e2cd73.el 7.0 MB/s | 9.8 MB 00:01 2024-03-29T04:05:15.629 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-03-29T04:05:18.151 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-29T04:05:18.152 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-29T04:05:18.166 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-29T04:05:18.167 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-29T04:05:18.168 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-29T04:05:18.169 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-29T04:05:18.171 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-29T04:05:18.174 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-29T04:05:18.175 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-29T04:05:18.254 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-03-29T04:05:18.266 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-29T04:05:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-03-29T04:05:18.266 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-29T04:05:18.266 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-03-29T04:05:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-29T04:05:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 6.6 k 2024-03-29T04:05:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 5.4 M 2024-03-29T04:05:18.266 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 890 k 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 145 k 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 1.5 M 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 141 k 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 3.5 M 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 7.4 M 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 49 k 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 9.8 M 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 48 M 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 264 k 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 762 k 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 221 k 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 27 k 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 18 k 2024-03-29T04:05:18.267 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-29T04:05:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-29T04:05:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-29T04:05:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 32 k 2024-03-29T04:05:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 746 k 2024-03-29T04:05:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 127 k 2024-03-29T04:05:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-29T04:05:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-29T04:05:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-29T04:05:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-03-29T04:05:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-03-29T04:05:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-03-29T04:05:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 162 k 2024-03-29T04:05:18.268 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 321 k 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 298 k 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 99 k 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 85 k 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.1 M 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 170 k 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-29T04:05:18.269 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-29T04:05:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-29T04:05:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-29T04:05:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-29T04:05:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-29T04:05:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-29T04:05:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.4 M 2024-03-29T04:05:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.1 M 2024-03-29T04:05:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-29T04:05:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-29T04:05:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-29T04:05:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-29T04:05:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-29T04:05:18.270 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 21 M 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 34 k 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 2.4 M 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 249 k 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 4.7 M 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 17 M 2024-03-29T04:05:18.271 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 16 k 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 25 k 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 163 k 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-29T04:05:18.272 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-29T04:05:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-29T04:05:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-29T04:05:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 500 k 2024-03-29T04:05:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-29T04:05:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 5.1 M 2024-03-29T04:05:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-29T04:05:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-29T04:05:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-29T04:05:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-29T04:05:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-29T04:05:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-29T04:05:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-29T04:05:18.273 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 45 k 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 134 k 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-29T04:05:18.274 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-29T04:05:18.275 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-29T04:05:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-29T04:05:18.277 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-29T04:05:18.278 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-29T04:05:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-29T04:05:18.279 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-03-29T04:05:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-03-29T04:05:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:05:18.279 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-03-29T04:05:18.279 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================================ 2024-03-29T04:05:18.279 INFO:teuthology.orchestra.run.smithi097.stdout:Install 150 Packages 2024-03-29T04:05:18.279 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 24 Packages 2024-03-29T04:05:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:05:18.280 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 226 M 2024-03-29T04:05:18.280 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-03-29T04:05:19.213 INFO:teuthology.orchestra.run.smithi097.stdout:(1/174): ceph-19.0.0-2520.g69e2cd73.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-03-29T04:05:19.580 INFO:teuthology.orchestra.run.smithi097.stdout:(2/174): ceph-base-19.0.0-2520.g69e2cd73.el9.x8 8.5 MB/s | 5.4 MB 00:00 2024-03-29T04:05:19.630 INFO:teuthology.orchestra.run.smithi097.stdout:(3/174): ceph-immutable-object-cache-19.0.0-252 2.8 MB/s | 145 kB 00:00 2024-03-29T04:05:19.797 INFO:teuthology.orchestra.run.smithi097.stdout:(4/174): ceph-mds-19.0.0-2520.g69e2cd73.el9.x86 14 MB/s | 2.4 MB 00:00 2024-03-29T04:05:19.914 INFO:teuthology.orchestra.run.smithi097.stdout:(5/174): ceph-mgr-19.0.0-2520.g69e2cd73.el9.x86 13 MB/s | 1.5 MB 00:00 2024-03-29T04:05:19.998 INFO:teuthology.orchestra.run.smithi097.stdout:(6/174): ceph-fuse-19.0.0-2520.g69e2cd73.el9.x8 1.1 MB/s | 890 kB 00:00 2024-03-29T04:05:20.281 INFO:teuthology.orchestra.run.smithi097.stdout:(7/174): ceph-mon-19.0.0-2520.g69e2cd73.el9.x86 13 MB/s | 4.7 MB 00:00 2024-03-29T04:05:20.303 INFO:teuthology.orchestra.run.smithi154.stdout:(11/174): cephfs-mirror-19.0.0-2520.g69e2cd73.e 32 kB/s | 221 kB 00:06 2024-03-29T04:05:20.395 INFO:teuthology.orchestra.run.smithi154.stdout:(12/174): cephfs-shell-19.0.0-2520.g69e2cd73.el 296 kB/s | 27 kB 00:00 2024-03-29T04:05:20.487 INFO:teuthology.orchestra.run.smithi154.stdout:(13/174): libcephfs-devel-19.0.0-2520.g69e2cd73 346 kB/s | 32 kB 00:00 2024-03-29T04:05:20.804 INFO:teuthology.orchestra.run.smithi154.stdout:(14/174): libcephfs2-19.0.0-2520.g69e2cd73.el9. 2.3 MB/s | 746 kB 00:00 2024-03-29T04:05:20.887 INFO:teuthology.orchestra.run.smithi154.stdout:(15/174): libcephsqlite-19.0.0-2520.g69e2cd73.e 1.9 MB/s | 163 kB 00:00 2024-03-29T04:05:20.979 INFO:teuthology.orchestra.run.smithi154.stdout:(16/174): librados-devel-19.0.0-2520.g69e2cd73. 1.3 MB/s | 127 kB 00:00 2024-03-29T04:05:21.179 INFO:teuthology.orchestra.run.smithi154.stdout:(17/174): libradosstriper1-19.0.0-2520.g69e2cd7 2.4 MB/s | 500 kB 00:00 2024-03-29T04:05:21.348 INFO:teuthology.orchestra.run.smithi097.stdout:(8/174): ceph-radosgw-19.0.0-2520.g69e2cd73.el9 9.2 MB/s | 9.8 MB 00:01 2024-03-29T04:05:21.432 INFO:teuthology.orchestra.run.smithi097.stdout:(9/174): ceph-selinux-19.0.0-2520.g69e2cd73.el9 302 kB/s | 25 kB 00:00 2024-03-29T04:05:22.613 INFO:teuthology.orchestra.run.smithi154.stdout:(18/174): librgw2-19.0.0-2520.g69e2cd73.el9.x86 3.6 MB/s | 5.1 MB 00:01 2024-03-29T04:05:22.713 INFO:teuthology.orchestra.run.smithi154.stdout:(19/174): python3-ceph-argparse-19.0.0-2520.g69 452 kB/s | 45 kB 00:00 2024-03-29T04:05:22.780 INFO:teuthology.orchestra.run.smithi154.stdout:(20/174): python3-ceph-common-19.0.0-2520.g69e2 2.0 MB/s | 134 kB 00:00 2024-03-29T04:05:22.847 INFO:teuthology.orchestra.run.smithi154.stdout:(21/174): python3-cephfs-19.0.0-2520.g69e2cd73. 2.4 MB/s | 162 kB 00:00 2024-03-29T04:05:22.931 INFO:teuthology.orchestra.run.smithi154.stdout:(22/174): python3-rados-19.0.0-2520.g69e2cd73.e 3.8 MB/s | 321 kB 00:00 2024-03-29T04:05:23.006 INFO:teuthology.orchestra.run.smithi154.stdout:(23/174): python3-rbd-19.0.0-2520.g69e2cd73.el9 3.9 MB/s | 298 kB 00:00 2024-03-29T04:05:23.056 INFO:teuthology.orchestra.run.smithi154.stdout:(24/174): python3-rgw-19.0.0-2520.g69e2cd73.el9 1.9 MB/s | 99 kB 00:00 2024-03-29T04:05:23.106 INFO:teuthology.orchestra.run.smithi154.stdout:(25/174): rbd-fuse-19.0.0-2520.g69e2cd73.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-29T04:05:23.590 INFO:teuthology.orchestra.run.smithi154.stdout:(26/174): rbd-mirror-19.0.0-2520.g69e2cd73.el9. 6.4 MB/s | 3.1 MB 00:00 2024-03-29T04:05:23.649 INFO:teuthology.orchestra.run.smithi154.stdout:(27/174): rbd-nbd-19.0.0-2520.g69e2cd73.el9.x86 2.9 MB/s | 170 kB 00:00 2024-03-29T04:05:23.699 INFO:teuthology.orchestra.run.smithi154.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2520.g 679 kB/s | 34 kB 00:00 2024-03-29T04:05:23.758 INFO:teuthology.orchestra.run.smithi154.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2520.g69e2cd7 2.4 MB/s | 141 kB 00:00 2024-03-29T04:05:24.391 INFO:teuthology.orchestra.run.smithi154.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2520.g69e2c 5.6 MB/s | 3.5 MB 00:00 2024-03-29T04:05:25.658 INFO:teuthology.orchestra.run.smithi154.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 5.9 MB/s | 7.4 MB 00:01 2024-03-29T04:05:25.725 INFO:teuthology.orchestra.run.smithi154.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2520.g69 3.7 MB/s | 249 kB 00:00 2024-03-29T04:05:25.783 INFO:teuthology.orchestra.run.smithi154.stdout:(33/174): ceph-mgr-rook-19.0.0-2520.g69e2cd73.e 846 kB/s | 49 kB 00:00 2024-03-29T04:05:25.834 INFO:teuthology.orchestra.run.smithi154.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2520.g6 329 kB/s | 16 kB 00:00 2024-03-29T04:05:25.901 INFO:teuthology.orchestra.run.smithi154.stdout:(35/174): ceph-volume-19.0.0-2520.g69e2cd73.el9 3.9 MB/s | 264 kB 00:00 2024-03-29T04:05:26.010 INFO:teuthology.orchestra.run.smithi154.stdout:(36/174): cephadm-19.0.0-2520.g69e2cd73.el9.noa 6.9 MB/s | 762 kB 00:00 2024-03-29T04:05:26.051 INFO:teuthology.orchestra.run.smithi154.stdout:(37/174): cephfs-top-19.0.0-2520.g69e2cd73.el9. 438 kB/s | 18 kB 00:00 2024-03-29T04:05:26.193 INFO:teuthology.orchestra.run.smithi154.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.6 MB/s | 662 kB 00:00 2024-03-29T04:05:26.215 INFO:teuthology.orchestra.run.smithi097.stdout:(10/174): ceph-test-19.0.0-2520.g69e2cd73.el9.x 10 MB/s | 48 MB 00:04 2024-03-29T04:05:26.265 INFO:teuthology.orchestra.run.smithi097.stdout:(11/174): cephfs-mirror-19.0.0-2520.g69e2cd73.e 4.4 MB/s | 221 kB 00:00 2024-03-29T04:05:26.319 INFO:teuthology.orchestra.run.smithi154.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 7.2 MB/s | 920 kB 00:00 2024-03-29T04:05:26.457 INFO:teuthology.orchestra.run.smithi097.stdout:(12/174): ceph-common-19.0.0-2520.g69e2cd73.el9 2.8 MB/s | 21 MB 00:07 2024-03-29T04:05:26.482 INFO:teuthology.orchestra.run.smithi097.stdout:(13/174): cephfs-shell-19.0.0-2520.g69e2cd73.el 125 kB/s | 27 kB 00:00 2024-03-29T04:05:26.494 INFO:teuthology.orchestra.run.smithi154.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 605 kB/s | 106 kB 00:00 2024-03-29T04:05:26.516 INFO:teuthology.orchestra.run.smithi097.stdout:(14/174): libcephfs-devel-19.0.0-2520.g69e2cd73 545 kB/s | 32 kB 00:00 2024-03-29T04:05:26.528 INFO:teuthology.orchestra.run.smithi154.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 9.0 MB/s | 307 kB 00:00 2024-03-29T04:05:26.561 INFO:teuthology.orchestra.run.smithi154.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 952 kB/s | 32 kB 00:00 2024-03-29T04:05:26.566 INFO:teuthology.orchestra.run.smithi097.stdout:(15/174): libcephfs2-19.0.0-2520.g69e2cd73.el9. 8.8 MB/s | 746 kB 00:00 2024-03-29T04:05:26.608 INFO:teuthology.orchestra.run.smithi097.stdout:(16/174): librados-devel-19.0.0-2520.g69e2cd73. 3.0 MB/s | 127 kB 00:00 2024-03-29T04:05:26.633 INFO:teuthology.orchestra.run.smithi097.stdout:(17/174): libcephsqlite-19.0.0-2520.g69e2cd73.e 1.4 MB/s | 163 kB 00:00 2024-03-29T04:05:26.662 INFO:teuthology.orchestra.run.smithi154.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-29T04:05:26.667 INFO:teuthology.orchestra.run.smithi097.stdout:(18/174): libradosstriper1-19.0.0-2520.g69e2cd7 8.4 MB/s | 500 kB 00:00 2024-03-29T04:05:26.696 INFO:teuthology.orchestra.run.smithi154.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 516 kB/s | 17 kB 00:00 2024-03-29T04:05:26.708 INFO:teuthology.orchestra.run.smithi097.stdout:(19/174): python3-ceph-argparse-19.0.0-2520.g69 1.1 MB/s | 45 kB 00:00 2024-03-29T04:05:26.729 INFO:teuthology.orchestra.run.smithi154.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 303 kB/s | 10 kB 00:00 2024-03-29T04:05:26.759 INFO:teuthology.orchestra.run.smithi097.stdout:(20/174): python3-ceph-common-19.0.0-2520.g69e2 2.6 MB/s | 134 kB 00:00 2024-03-29T04:05:26.762 INFO:teuthology.orchestra.run.smithi154.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 159 kB 00:00 2024-03-29T04:05:26.796 INFO:teuthology.orchestra.run.smithi154.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-29T04:05:26.826 INFO:teuthology.orchestra.run.smithi097.stdout:(21/174): python3-cephfs-19.0.0-2520.g69e2cd73. 2.4 MB/s | 162 kB 00:00 2024-03-29T04:05:26.838 INFO:teuthology.orchestra.run.smithi154.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 661 kB 00:00 2024-03-29T04:05:26.880 INFO:teuthology.orchestra.run.smithi154.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 578 kB 00:00 2024-03-29T04:05:26.885 INFO:teuthology.orchestra.run.smithi097.stdout:(22/174): python3-rados-19.0.0-2520.g69e2cd73.e 5.4 MB/s | 321 kB 00:00 2024-03-29T04:05:26.914 INFO:teuthology.orchestra.run.smithi154.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-03-29T04:05:26.952 INFO:teuthology.orchestra.run.smithi097.stdout:(23/174): python3-rbd-19.0.0-2520.g69e2cd73.el9 4.4 MB/s | 298 kB 00:00 2024-03-29T04:05:26.955 INFO:teuthology.orchestra.run.smithi154.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-29T04:05:26.994 INFO:teuthology.orchestra.run.smithi097.stdout:(24/174): python3-rgw-19.0.0-2520.g69e2cd73.el9 2.3 MB/s | 99 kB 00:00 2024-03-29T04:05:27.044 INFO:teuthology.orchestra.run.smithi097.stdout:(25/174): rbd-fuse-19.0.0-2520.g69e2cd73.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-29T04:05:27.123 INFO:teuthology.orchestra.run.smithi154.stdout:(52/174): ceph-osd-19.0.0-2520.g69e2cd73.el9.x8 801 kB/s | 17 MB 00:21 2024-03-29T04:05:27.148 INFO:teuthology.orchestra.run.smithi154.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 973 kB/s | 187 kB 00:00 2024-03-29T04:05:27.173 INFO:teuthology.orchestra.run.smithi154.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-29T04:05:27.199 INFO:teuthology.orchestra.run.smithi154.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 683 kB/s | 34 kB 00:00 2024-03-29T04:05:27.299 INFO:teuthology.orchestra.run.smithi154.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-29T04:05:27.303 INFO:teuthology.orchestra.run.smithi097.stdout:(26/174): rbd-mirror-19.0.0-2520.g69e2cd73.el9. 12 MB/s | 3.1 MB 00:00 2024-03-29T04:05:27.341 INFO:teuthology.orchestra.run.smithi154.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2024-03-29T04:05:27.344 INFO:teuthology.orchestra.run.smithi097.stdout:(27/174): rbd-nbd-19.0.0-2520.g69e2cd73.el9.x86 4.0 MB/s | 170 kB 00:00 2024-03-29T04:05:27.366 INFO:teuthology.orchestra.run.smithi154.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 817 kB 00:00 2024-03-29T04:05:27.378 INFO:teuthology.orchestra.run.smithi097.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2520.g 1.0 MB/s | 34 kB 00:00 2024-03-29T04:05:27.392 INFO:teuthology.orchestra.run.smithi154.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 132 kB 00:00 2024-03-29T04:05:27.420 INFO:teuthology.orchestra.run.smithi097.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2520.g69e2cd7 3.3 MB/s | 141 kB 00:00 2024-03-29T04:05:27.509 INFO:teuthology.orchestra.run.smithi154.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 40 MB/s | 5.7 MB 00:00 2024-03-29T04:05:27.543 INFO:teuthology.orchestra.run.smithi154.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.8 MB/s | 1.0 MB 00:00 2024-03-29T04:05:27.567 INFO:teuthology.orchestra.run.smithi154.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-29T04:05:27.684 INFO:teuthology.orchestra.run.smithi154.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 41 MB/s | 5.8 MB 00:00 2024-03-29T04:05:27.695 INFO:teuthology.orchestra.run.smithi097.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2520.g69e2c 13 MB/s | 3.5 MB 00:00 2024-03-29T04:05:27.710 INFO:teuthology.orchestra.run.smithi154.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 175 kB/s | 24 kB 00:00 2024-03-29T04:05:27.735 INFO:teuthology.orchestra.run.smithi154.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 362 kB/s | 18 kB 00:00 2024-03-29T04:05:27.754 INFO:teuthology.orchestra.run.smithi097.stdout:(31/174): librgw2-19.0.0-2520.g69e2cd73.el9.x86 4.6 MB/s | 5.1 MB 00:01 2024-03-29T04:05:27.760 INFO:teuthology.orchestra.run.smithi154.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-29T04:05:27.786 INFO:teuthology.orchestra.run.smithi154.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 631 kB/s | 32 kB 00:00 2024-03-29T04:05:27.887 INFO:teuthology.orchestra.run.smithi097.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2520.g69 1.8 MB/s | 249 kB 00:00 2024-03-29T04:05:27.927 INFO:teuthology.orchestra.run.smithi154.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2024-03-29T04:05:27.953 INFO:teuthology.orchestra.run.smithi154.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-29T04:05:27.978 INFO:teuthology.orchestra.run.smithi154.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-29T04:05:28.003 INFO:teuthology.orchestra.run.smithi154.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-29T04:05:28.029 INFO:teuthology.orchestra.run.smithi154.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 821 kB/s | 41 kB 00:00 2024-03-29T04:05:28.046 INFO:teuthology.orchestra.run.smithi097.stdout:(33/174): ceph-osd-19.0.0-2520.g69e2cd73.el9.x8 2.1 MB/s | 17 MB 00:08 2024-03-29T04:05:28.054 INFO:teuthology.orchestra.run.smithi154.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-29T04:05:28.071 INFO:teuthology.orchestra.run.smithi097.stdout:(34/174): ceph-mgr-rook-19.0.0-2520.g69e2cd73.e 269 kB/s | 49 kB 00:00 2024-03-29T04:05:28.079 INFO:teuthology.orchestra.run.smithi154.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-29T04:05:28.104 INFO:teuthology.orchestra.run.smithi154.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-29T04:05:28.113 INFO:teuthology.orchestra.run.smithi097.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2520.g6 247 kB/s | 16 kB 00:00 2024-03-29T04:05:28.130 INFO:teuthology.orchestra.run.smithi154.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 946 kB/s | 47 kB 00:00 2024-03-29T04:05:28.155 INFO:teuthology.orchestra.run.smithi154.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 853 kB/s | 43 kB 00:00 2024-03-29T04:05:28.238 INFO:teuthology.orchestra.run.smithi097.stdout:(36/174): ceph-volume-19.0.0-2520.g69e2cd73.el9 1.5 MB/s | 264 kB 00:00 2024-03-29T04:05:28.322 INFO:teuthology.orchestra.run.smithi154.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 235 kB/s | 39 kB 00:00 2024-03-29T04:05:28.347 INFO:teuthology.orchestra.run.smithi097.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-03-29T04:05:28.373 INFO:teuthology.orchestra.run.smithi097.stdout:(38/174): cephfs-top-19.0.0-2520.g69e2cd73.el9. 136 kB/s | 18 kB 00:00 2024-03-29T04:05:28.473 INFO:teuthology.orchestra.run.smithi154.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-03-29T04:05:28.473 INFO:teuthology.orchestra.run.smithi097.stdout:(39/174): cephadm-19.0.0-2520.g69e2cd73.el9.noa 2.1 MB/s | 762 kB 00:00 2024-03-29T04:05:28.498 INFO:teuthology.orchestra.run.smithi154.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 236 kB/s | 41 kB 00:00 2024-03-29T04:05:28.523 INFO:teuthology.orchestra.run.smithi154.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2024-03-29T04:05:28.548 INFO:teuthology.orchestra.run.smithi154.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 695 kB/s | 35 kB 00:00 2024-03-29T04:05:28.565 INFO:teuthology.orchestra.run.smithi097.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.7 MB/s | 920 kB 00:00 2024-03-29T04:05:28.590 INFO:teuthology.orchestra.run.smithi154.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 690 kB/s | 46 kB 00:00 2024-03-29T04:05:28.615 INFO:teuthology.orchestra.run.smithi097.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-03-29T04:05:28.616 INFO:teuthology.orchestra.run.smithi154.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 300 kB 00:00 2024-03-29T04:05:28.640 INFO:teuthology.orchestra.run.smithi097.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 638 kB/s | 106 kB 00:00 2024-03-29T04:05:28.641 INFO:teuthology.orchestra.run.smithi154.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-29T04:05:28.665 INFO:teuthology.orchestra.run.smithi097.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 307 kB 00:00 2024-03-29T04:05:28.667 INFO:teuthology.orchestra.run.smithi154.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-29T04:05:28.691 INFO:teuthology.orchestra.run.smithi097.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 420 kB/s | 32 kB 00:00 2024-03-29T04:05:28.691 INFO:teuthology.orchestra.run.smithi154.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 878 kB/s | 44 kB 00:00 2024-03-29T04:05:28.717 INFO:teuthology.orchestra.run.smithi154.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-29T04:05:28.749 INFO:teuthology.orchestra.run.smithi097.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 206 kB/s | 17 kB 00:00 2024-03-29T04:05:28.750 INFO:teuthology.orchestra.run.smithi154.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-03-29T04:05:28.793 INFO:teuthology.orchestra.run.smithi154.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-03-29T04:05:28.818 INFO:teuthology.orchestra.run.smithi154.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-29T04:05:28.833 INFO:teuthology.orchestra.run.smithi097.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 71 kB/s | 10 kB 00:00 2024-03-29T04:05:28.843 INFO:teuthology.orchestra.run.smithi154.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-29T04:05:28.869 INFO:teuthology.orchestra.run.smithi154.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 187 kB 00:00 2024-03-29T04:05:28.921 INFO:teuthology.orchestra.run.smithi097.stdout:(47/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-03-29T04:05:28.950 INFO:teuthology.orchestra.run.smithi097.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 797 kB/s | 159 kB 00:00 2024-03-29T04:05:28.975 INFO:teuthology.orchestra.run.smithi097.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 313 kB/s | 44 kB 00:00 2024-03-29T04:05:29.025 INFO:teuthology.orchestra.run.smithi097.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-29T04:05:29.051 INFO:teuthology.orchestra.run.smithi097.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 578 kB 00:00 2024-03-29T04:05:29.052 INFO:teuthology.orchestra.run.smithi154.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 899 kB/s | 187 kB 00:00 2024-03-29T04:05:29.085 INFO:teuthology.orchestra.run.smithi097.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 427 kB/s | 46 kB 00:00 2024-03-29T04:05:29.110 INFO:teuthology.orchestra.run.smithi097.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-03-29T04:05:29.135 INFO:teuthology.orchestra.run.smithi097.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-29T04:05:29.136 INFO:teuthology.orchestra.run.smithi154.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-03-29T04:05:29.160 INFO:teuthology.orchestra.run.smithi097.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-29T04:05:29.169 INFO:teuthology.orchestra.run.smithi154.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-03-29T04:05:29.185 INFO:teuthology.orchestra.run.smithi097.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 458 kB/s | 34 kB 00:00 2024-03-29T04:05:29.195 INFO:teuthology.orchestra.run.smithi154.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-29T04:05:29.294 INFO:teuthology.orchestra.run.smithi097.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2024-03-29T04:05:29.386 INFO:teuthology.orchestra.run.smithi097.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 4.0 MB/s | 817 kB 00:00 2024-03-29T04:05:29.437 INFO:teuthology.orchestra.run.smithi154.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 80 kB/s | 21 kB 00:00 2024-03-29T04:05:29.444 INFO:teuthology.orchestra.run.smithi097.stdout:(59/174): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-03-29T04:05:29.469 INFO:teuthology.orchestra.run.smithi097.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 762 kB/s | 132 kB 00:00 2024-03-29T04:05:29.470 INFO:teuthology.orchestra.run.smithi154.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 92 kB/s | 25 kB 00:00 2024-03-29T04:05:29.578 INFO:teuthology.orchestra.run.smithi097.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.7 MB 00:00 2024-03-29T04:05:29.603 INFO:teuthology.orchestra.run.smithi097.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 595 kB/s | 78 kB 00:00 2024-03-29T04:05:29.637 INFO:teuthology.orchestra.run.smithi097.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-03-29T04:05:29.779 INFO:teuthology.orchestra.run.smithi097.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 5.8 MB 00:00 2024-03-29T04:05:29.804 INFO:teuthology.orchestra.run.smithi097.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 123 kB/s | 24 kB 00:00 2024-03-29T04:05:29.829 INFO:teuthology.orchestra.run.smithi097.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 95 kB/s | 18 kB 00:00 2024-03-29T04:05:29.855 INFO:teuthology.orchestra.run.smithi097.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-29T04:05:29.880 INFO:teuthology.orchestra.run.smithi097.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 636 kB/s | 32 kB 00:00 2024-03-29T04:05:29.980 INFO:teuthology.orchestra.run.smithi097.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-29T04:05:30.005 INFO:teuthology.orchestra.run.smithi097.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-03-29T04:05:30.030 INFO:teuthology.orchestra.run.smithi097.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-03-29T04:05:30.054 INFO:teuthology.orchestra.run.smithi154.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 421 kB/s | 245 kB 00:00 2024-03-29T04:05:30.056 INFO:teuthology.orchestra.run.smithi097.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 920 kB/s | 69 kB 00:00 2024-03-29T04:05:30.081 INFO:teuthology.orchestra.run.smithi097.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 821 kB/s | 41 kB 00:00 2024-03-29T04:05:30.115 INFO:teuthology.orchestra.run.smithi097.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-03-29T04:05:30.140 INFO:teuthology.orchestra.run.smithi097.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-03-29T04:05:30.165 INFO:teuthology.orchestra.run.smithi097.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-29T04:05:30.190 INFO:teuthology.orchestra.run.smithi097.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 631 kB/s | 47 kB 00:00 2024-03-29T04:05:30.215 INFO:teuthology.orchestra.run.smithi097.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 572 kB/s | 43 kB 00:00 2024-03-29T04:05:30.241 INFO:teuthology.orchestra.run.smithi097.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 776 kB/s | 39 kB 00:00 2024-03-29T04:05:30.366 INFO:teuthology.orchestra.run.smithi097.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 273 kB/s | 41 kB 00:00 2024-03-29T04:05:30.433 INFO:teuthology.orchestra.run.smithi097.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2024-03-29T04:05:30.583 INFO:teuthology.orchestra.run.smithi097.stdout:(82/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-03-29T04:05:30.608 INFO:teuthology.orchestra.run.smithi097.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 145 kB/s | 35 kB 00:00 2024-03-29T04:05:30.634 INFO:teuthology.orchestra.run.smithi097.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 232 kB/s | 46 kB 00:00 2024-03-29T04:05:30.659 INFO:teuthology.orchestra.run.smithi097.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-29T04:05:30.685 INFO:teuthology.orchestra.run.smithi097.stdout:(86/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-03-29T04:05:30.709 INFO:teuthology.orchestra.run.smithi097.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 830 kB/s | 63 kB 00:00 2024-03-29T04:05:30.735 INFO:teuthology.orchestra.run.smithi097.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 585 kB/s | 44 kB 00:00 2024-03-29T04:05:30.885 INFO:teuthology.orchestra.run.smithi097.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 449 kB/s | 79 kB 00:00 2024-03-29T04:05:30.910 INFO:teuthology.orchestra.run.smithi097.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 241 kB 00:00 2024-03-29T04:05:30.952 INFO:teuthology.orchestra.run.smithi097.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 5.3 MB/s | 1.1 MB 00:00 2024-03-29T04:05:30.977 INFO:teuthology.orchestra.run.smithi097.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 1.9 MB/s | 124 kB 00:00 2024-03-29T04:05:31.003 INFO:teuthology.orchestra.run.smithi097.stdout:(93/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-29T04:05:31.028 INFO:teuthology.orchestra.run.smithi097.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-29T04:05:31.062 INFO:teuthology.orchestra.run.smithi154.stdout:(101/174): ceph-test-19.0.0-2520.g69e2cd73.el9. 2.6 MB/s | 48 MB 00:18 2024-03-29T04:05:31.096 INFO:teuthology.orchestra.run.smithi154.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 238 kB/s | 248 kB 00:01 2024-03-29T04:05:31.263 INFO:teuthology.orchestra.run.smithi154.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-03-29T04:05:31.405 INFO:teuthology.orchestra.run.smithi154.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-03-29T04:05:31.536 INFO:teuthology.orchestra.run.smithi097.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 335 kB/s | 187 kB 00:00 2024-03-29T04:05:31.538 INFO:teuthology.orchestra.run.smithi154.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 197 kB/s | 94 kB 00:00 2024-03-29T04:05:31.562 INFO:teuthology.orchestra.run.smithi097.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 366 kB/s | 195 kB 00:00 2024-03-29T04:05:31.645 INFO:teuthology.orchestra.run.smithi097.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-03-29T04:05:31.687 INFO:teuthology.orchestra.run.smithi097.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 172 kB/s | 21 kB 00:00 2024-03-29T04:05:31.796 INFO:teuthology.orchestra.run.smithi097.stdout:(99/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-03-29T04:05:31.821 INFO:teuthology.orchestra.run.smithi097.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 145 kB/s | 25 kB 00:00 2024-03-29T04:05:31.980 INFO:teuthology.orchestra.run.smithi097.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.3 MB/s | 245 kB 00:00 2024-03-29T04:05:32.038 INFO:teuthology.orchestra.run.smithi097.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-29T04:05:32.064 INFO:teuthology.orchestra.run.smithi097.stdout:(103/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.0 MB/s | 248 kB 00:00 2024-03-29T04:05:32.106 INFO:teuthology.orchestra.run.smithi154.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 782 kB/s | 548 kB 00:00 2024-03-29T04:05:32.139 INFO:teuthology.orchestra.run.smithi097.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-03-29T04:05:32.147 INFO:teuthology.orchestra.run.smithi154.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 507 kB/s | 309 kB 00:00 2024-03-29T04:05:32.198 INFO:teuthology.orchestra.run.smithi154.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-03-29T04:05:32.247 INFO:teuthology.orchestra.run.smithi097.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-03-29T04:05:32.340 INFO:teuthology.orchestra.run.smithi154.stdout:(109/174): openssl-devel-3.0.7-27.el9.x86_64.rp 1.4 MB/s | 4.1 MB 00:02 2024-03-29T04:05:32.357 INFO:teuthology.orchestra.run.smithi154.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 163 kB/s | 26 kB 00:00 2024-03-29T04:05:32.414 INFO:teuthology.orchestra.run.smithi097.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-03-29T04:05:32.473 INFO:teuthology.orchestra.run.smithi097.stdout:(107/174): openssl-devel-3.0.7-27.el9.x86_64.rp 5.3 MB/s | 4.1 MB 00:00 2024-03-29T04:05:32.474 INFO:teuthology.orchestra.run.smithi154.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 580 kB/s | 67 kB 00:00 2024-03-29T04:05:32.498 INFO:teuthology.orchestra.run.smithi097.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 232 kB/s | 19 kB 00:00 2024-03-29T04:05:32.523 INFO:teuthology.orchestra.run.smithi097.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-03-29T04:05:32.565 INFO:teuthology.orchestra.run.smithi097.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-03-29T04:05:32.591 INFO:teuthology.orchestra.run.smithi154.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-03-29T04:05:32.599 INFO:teuthology.orchestra.run.smithi097.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 647 kB/s | 49 kB 00:00 2024-03-29T04:05:32.649 INFO:teuthology.orchestra.run.smithi097.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 807 kB/s | 67 kB 00:00 2024-03-29T04:05:32.650 INFO:teuthology.orchestra.run.smithi154.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-03-29T04:05:32.699 INFO:teuthology.orchestra.run.smithi097.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-03-29T04:05:32.758 INFO:teuthology.orchestra.run.smithi154.stdout:(114/174): liboath-2.6.7-2.el9.x86_64.rpm 116 kB/s | 49 kB 00:00 2024-03-29T04:05:32.791 INFO:teuthology.orchestra.run.smithi154.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-03-29T04:05:32.817 INFO:teuthology.orchestra.run.smithi154.stdout:(116/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-03-29T04:05:32.884 INFO:teuthology.orchestra.run.smithi154.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 347 kB/s | 43 kB 00:00 2024-03-29T04:05:32.901 INFO:teuthology.orchestra.run.smithi154.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-03-29T04:05:32.908 INFO:teuthology.orchestra.run.smithi097.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 838 kB 00:00 2024-03-29T04:05:32.918 INFO:teuthology.orchestra.run.smithi154.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-03-29T04:05:32.933 INFO:teuthology.orchestra.run.smithi097.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-03-29T04:05:32.985 INFO:teuthology.orchestra.run.smithi154.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-29T04:05:32.992 INFO:teuthology.orchestra.run.smithi097.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-03-29T04:05:33.017 INFO:teuthology.orchestra.run.smithi097.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 386 kB/s | 32 kB 00:00 2024-03-29T04:05:33.018 INFO:teuthology.orchestra.run.smithi154.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-03-29T04:05:33.068 INFO:teuthology.orchestra.run.smithi097.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-03-29T04:05:33.077 INFO:teuthology.orchestra.run.smithi154.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 385 kB/s | 35 kB 00:00 2024-03-29T04:05:33.093 INFO:teuthology.orchestra.run.smithi097.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-03-29T04:05:33.127 INFO:teuthology.orchestra.run.smithi154.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-03-29T04:05:33.152 INFO:teuthology.orchestra.run.smithi097.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-29T04:05:33.177 INFO:teuthology.orchestra.run.smithi097.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-03-29T04:05:33.219 INFO:teuthology.orchestra.run.smithi097.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 528 kB/s | 35 kB 00:00 2024-03-29T04:05:33.227 INFO:teuthology.orchestra.run.smithi154.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-03-29T04:05:33.244 INFO:teuthology.orchestra.run.smithi097.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-03-29T04:05:33.253 INFO:teuthology.orchestra.run.smithi154.stdout:(125/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 905 kB/s | 303 kB 00:00 2024-03-29T04:05:33.278 INFO:teuthology.orchestra.run.smithi154.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 960 kB/s | 144 kB 00:00 2024-03-29T04:05:33.328 INFO:teuthology.orchestra.run.smithi154.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-03-29T04:05:33.336 INFO:teuthology.orchestra.run.smithi097.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-03-29T04:05:33.353 INFO:teuthology.orchestra.run.smithi154.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-03-29T04:05:33.379 INFO:teuthology.orchestra.run.smithi154.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-03-29T04:05:33.429 INFO:teuthology.orchestra.run.smithi154.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-03-29T04:05:33.436 INFO:teuthology.orchestra.run.smithi097.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-03-29T04:05:33.454 INFO:teuthology.orchestra.run.smithi154.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-03-29T04:05:33.512 INFO:teuthology.orchestra.run.smithi097.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 7.0 MB/s | 2.0 MB 00:00 2024-03-29T04:05:33.521 INFO:teuthology.orchestra.run.smithi154.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-03-29T04:05:33.537 INFO:teuthology.orchestra.run.smithi097.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-03-29T04:05:33.546 INFO:teuthology.orchestra.run.smithi154.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 394 kB/s | 46 kB 00:00 2024-03-29T04:05:33.572 INFO:teuthology.orchestra.run.smithi154.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 672 kB/s | 79 kB 00:00 2024-03-29T04:05:33.604 INFO:teuthology.orchestra.run.smithi097.stdout:(128/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-03-29T04:05:33.614 INFO:teuthology.orchestra.run.smithi154.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-03-29T04:05:33.629 INFO:teuthology.orchestra.run.smithi097.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-03-29T04:05:33.647 INFO:teuthology.orchestra.run.smithi154.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-03-29T04:05:33.654 INFO:teuthology.orchestra.run.smithi097.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-03-29T04:05:33.664 INFO:teuthology.orchestra.run.smithi154.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-03-29T04:05:33.679 INFO:teuthology.orchestra.run.smithi097.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-03-29T04:05:33.707 INFO:teuthology.orchestra.run.smithi154.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 971 kB/s | 90 kB 00:00 2024-03-29T04:05:33.713 INFO:teuthology.orchestra.run.smithi097.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 794 kB/s | 46 kB 00:00 2024-03-29T04:05:33.738 INFO:teuthology.orchestra.run.smithi097.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-29T04:05:33.740 INFO:teuthology.orchestra.run.smithi154.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-03-29T04:05:33.757 INFO:teuthology.orchestra.run.smithi154.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 332 kB/s | 31 kB 00:00 2024-03-29T04:05:33.772 INFO:teuthology.orchestra.run.smithi097.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-03-29T04:05:33.797 INFO:teuthology.orchestra.run.smithi097.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 690 kB/s | 58 kB 00:00 2024-03-29T04:05:33.816 INFO:teuthology.orchestra.run.smithi154.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-03-29T04:05:33.822 INFO:teuthology.orchestra.run.smithi097.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-29T04:05:33.833 INFO:teuthology.orchestra.run.smithi154.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-03-29T04:05:33.848 INFO:teuthology.orchestra.run.smithi097.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-03-29T04:05:33.849 INFO:teuthology.orchestra.run.smithi154.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-03-29T04:05:33.865 INFO:teuthology.orchestra.run.smithi097.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-03-29T04:05:33.890 INFO:teuthology.orchestra.run.smithi097.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 421 kB/s | 28 kB 00:00 2024-03-29T04:05:33.908 INFO:teuthology.orchestra.run.smithi154.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 556 kB/s | 51 kB 00:00 2024-03-29T04:05:33.915 INFO:teuthology.orchestra.run.smithi097.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 458 kB/s | 31 kB 00:00 2024-03-29T04:05:33.933 INFO:teuthology.orchestra.run.smithi154.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-29T04:05:33.940 INFO:teuthology.orchestra.run.smithi097.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-29T04:05:33.959 INFO:teuthology.orchestra.run.smithi154.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 825 kB/s | 90 kB 00:00 2024-03-29T04:05:33.965 INFO:teuthology.orchestra.run.smithi097.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-03-29T04:05:33.991 INFO:teuthology.orchestra.run.smithi097.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-03-29T04:05:34.017 INFO:teuthology.orchestra.run.smithi097.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-03-29T04:05:34.034 INFO:teuthology.orchestra.run.smithi154.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-03-29T04:05:34.041 INFO:teuthology.orchestra.run.smithi097.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-29T04:05:34.051 INFO:teuthology.orchestra.run.smithi154.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-03-29T04:05:34.067 INFO:teuthology.orchestra.run.smithi097.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-29T04:05:34.085 INFO:teuthology.orchestra.run.smithi154.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-03-29T04:05:34.109 INFO:teuthology.orchestra.run.smithi097.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-03-29T04:05:34.126 INFO:teuthology.orchestra.run.smithi097.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-03-29T04:05:34.151 INFO:teuthology.orchestra.run.smithi097.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-03-29T04:05:34.202 INFO:teuthology.orchestra.run.smithi154.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-03-29T04:05:34.302 INFO:teuthology.orchestra.run.smithi097.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-03-29T04:05:34.310 INFO:teuthology.orchestra.run.smithi154.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 21 MB/s | 2.3 MB 00:00 2024-03-29T04:05:34.419 INFO:teuthology.orchestra.run.smithi154.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 17 MB/s | 1.8 MB 00:00 2024-03-29T04:05:34.444 INFO:teuthology.orchestra.run.smithi154.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.6 MB/s | 41 kB 00:00 2024-03-29T04:05:34.495 INFO:teuthology.orchestra.run.smithi154.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.9 MB/s | 247 kB 00:00 2024-03-29T04:05:34.553 INFO:teuthology.orchestra.run.smithi154.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-03-29T04:05:34.669 INFO:teuthology.orchestra.run.smithi097.stdout:(151/174): librbd1-19.0.0-2520.g69e2cd73.el9.x8 6.1 MB/s | 3.1 MB 00:00 2024-03-29T04:05:34.670 INFO:teuthology.orchestra.run.smithi154.stdout:(156/174): librados2-19.0.0-2520.g69e2cd73.el9. 5.5 MB/s | 3.4 MB 00:00 2024-03-29T04:05:34.695 INFO:teuthology.orchestra.run.smithi154.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 503 kB/s | 71 kB 00:00 2024-03-29T04:05:34.710 INFO:teuthology.orchestra.run.smithi097.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 5.6 MB/s | 2.3 MB 00:00 2024-03-29T04:05:34.729 INFO:teuthology.orchestra.run.smithi154.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 624 kB/s | 36 kB 00:00 2024-03-29T04:05:34.777 INFO:teuthology.orchestra.run.smithi097.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 17 MB/s | 1.8 MB 00:00 2024-03-29T04:05:34.802 INFO:teuthology.orchestra.run.smithi097.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 454 kB/s | 41 kB 00:00 2024-03-29T04:05:34.804 INFO:teuthology.orchestra.run.smithi154.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-03-29T04:05:34.829 INFO:teuthology.orchestra.run.smithi154.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.5 MB/s | 256 kB 00:00 2024-03-29T04:05:34.836 INFO:teuthology.orchestra.run.smithi097.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.1 MB/s | 247 kB 00:00 2024-03-29T04:05:34.861 INFO:teuthology.orchestra.run.smithi097.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-03-29T04:05:34.871 INFO:teuthology.orchestra.run.smithi154.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 344 kB/s | 23 kB 00:00 2024-03-29T04:05:34.905 INFO:teuthology.orchestra.run.smithi154.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-03-29T04:05:34.989 INFO:teuthology.orchestra.run.smithi154.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-29T04:05:35.022 INFO:teuthology.orchestra.run.smithi154.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 155 kB 00:00 2024-03-29T04:05:35.072 INFO:teuthology.orchestra.run.smithi154.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 765 kB/s | 64 kB 00:00 2024-03-29T04:05:35.106 INFO:teuthology.orchestra.run.smithi154.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 344 kB/s | 29 kB 00:00 2024-03-29T04:05:35.178 INFO:teuthology.orchestra.run.smithi097.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 209 kB/s | 71 kB 00:00 2024-03-29T04:05:35.220 INFO:teuthology.orchestra.run.smithi097.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 102 kB/s | 36 kB 00:00 2024-03-29T04:05:35.398 INFO:teuthology.orchestra.run.smithi154.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 3.7 MB/s | 1.2 MB 00:00 2024-03-29T04:05:35.515 INFO:teuthology.orchestra.run.smithi154.stdout:(168/174): librbd1-19.0.0-2520.g69e2cd73.el9.x8 2.2 MB/s | 3.1 MB 00:01 2024-03-29T04:05:35.529 INFO:teuthology.orchestra.run.smithi097.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 549 kB/s | 192 kB 00:00 2024-03-29T04:05:35.599 INFO:teuthology.orchestra.run.smithi154.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.4 MB/s | 2.2 MB 00:00 2024-03-29T04:05:35.637 INFO:teuthology.orchestra.run.smithi097.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 213 kB/s | 23 kB 00:00 2024-03-29T04:05:35.649 INFO:teuthology.orchestra.run.smithi154.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 626 kB/s | 31 kB 00:00 2024-03-29T04:05:35.663 INFO:teuthology.orchestra.run.smithi097.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 578 kB/s | 256 kB 00:00 2024-03-29T04:05:35.708 INFO:teuthology.orchestra.run.smithi154.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.4 MB/s | 2.3 MB 00:00 2024-03-29T04:05:35.763 INFO:teuthology.orchestra.run.smithi097.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 865 kB/s | 108 kB 00:00 2024-03-29T04:05:35.774 INFO:teuthology.orchestra.run.smithi154.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.6 MB/s | 1.1 MB 00:00 2024-03-29T04:05:35.788 INFO:teuthology.orchestra.run.smithi097.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 155 kB 00:00 2024-03-29T04:05:35.833 INFO:teuthology.orchestra.run.smithi154.stdout:(173/174): util-linux-core-2.37.4-18.el9.x86_64 1.4 MB/s | 465 kB 00:00 2024-03-29T04:05:35.947 INFO:teuthology.orchestra.run.smithi097.stdout:(164/174): librados2-19.0.0-2520.g69e2cd73.el9. 1.9 MB/s | 3.4 MB 00:01 2024-03-29T04:05:35.972 INFO:teuthology.orchestra.run.smithi097.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 348 kB/s | 64 kB 00:00 2024-03-29T04:05:35.997 INFO:teuthology.orchestra.run.smithi097.stdout:(166/174): libmount-2.37.4-18.el9.x86_64.rpm 579 kB/s | 136 kB 00:00 2024-03-29T04:05:36.125 INFO:teuthology.orchestra.run.smithi154.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 178 kB/s | 74 kB 00:00 2024-03-29T04:05:36.131 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:05:36.132 INFO:teuthology.orchestra.run.smithi154.stdout:Total 6.4 MB/s | 226 MB 00:35 2024-03-29T04:05:36.231 INFO:teuthology.orchestra.run.smithi097.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 101 kB/s | 29 kB 00:00 2024-03-29T04:05:36.348 INFO:teuthology.orchestra.run.smithi097.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.2 MB/s | 1.2 MB 00:00 2024-03-29T04:05:36.473 INFO:teuthology.orchestra.run.smithi097.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.6 MB/s | 465 kB 00:00 2024-03-29T04:05:36.511 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-03-29T04:05:36.515 INFO:teuthology.orchestra.run.smithi097.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.2 MB/s | 2.2 MB 00:00 2024-03-29T04:05:36.574 INFO:teuthology.orchestra.run.smithi097.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 310 kB/s | 31 kB 00:00 2024-03-29T04:05:36.632 INFO:teuthology.orchestra.run.smithi097.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.2 MB/s | 1.1 MB 00:00 2024-03-29T04:05:36.724 INFO:teuthology.orchestra.run.smithi097.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 495 kB/s | 74 kB 00:00 2024-03-29T04:05:36.790 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-03-29T04:05:36.790 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-03-29T04:05:36.949 INFO:teuthology.orchestra.run.smithi097.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 2.3 MB 00:00 2024-03-29T04:05:36.953 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:05:36.953 INFO:teuthology.orchestra.run.smithi097.stdout:Total 12 MB/s | 226 MB 00:18 2024-03-29T04:05:37.320 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-03-29T04:05:37.597 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-03-29T04:05:37.597 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-03-29T04:05:38.818 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-03-29T04:05:38.818 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-03-29T04:05:39.628 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-03-29T04:05:39.628 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-03-29T04:05:45.729 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-03-29T04:05:45.838 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-29T04:05:45.872 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-29T04:05:45.910 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-29T04:05:45.921 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-29T04:05:45.960 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-29T04:05:46.015 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-29T04:05:46.047 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-29T04:05:46.074 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-29T04:05:46.125 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-29T04:05:46.468 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-29T04:05:46.506 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-03-29T04:05:46.534 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:05:46.604 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-29T04:05:46.639 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-29T04:05:46.658 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:05:46.677 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-29T04:05:46.686 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-29T04:05:46.711 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:05:46.726 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-29T04:05:46.785 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-29T04:05:46.793 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:05:46.830 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-29T04:05:46.833 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 12/198 2024-03-29T04:05:46.855 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-29T04:05:46.878 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-29T04:05:46.901 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-29T04:05:47.058 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-29T04:05:47.112 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-29T04:05:47.141 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:05:47.228 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:05:47.241 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-29T04:05:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 17/198 2024-03-29T04:05:47.298 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:05:47.299 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 17/198 2024-03-29T04:05:47.346 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-29T04:05:47.368 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-29T04:05:47.384 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-29T04:05:47.421 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:05:47.444 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:05:47.519 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:05:47.528 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-29T04:05:47.558 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-29T04:05:47.565 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 12/198 2024-03-29T04:05:47.589 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-29T04:05:47.609 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-29T04:05:47.771 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-29T04:05:47.784 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-29T04:05:47.829 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-29T04:05:47.849 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:05:47.921 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-29T04:05:47.953 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:05:47.975 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 17/198 2024-03-29T04:05:47.980 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-29T04:05:48.040 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 17/198 2024-03-29T04:05:48.070 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-29T04:05:48.072 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:05:48.093 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-29T04:05:48.109 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-29T04:05:48.123 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:05:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-29T04:05:48.326 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:05:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-29T04:05:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-29T04:05:48.437 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-29T04:05:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-29T04:05:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-29T04:05:48.572 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-29T04:05:48.597 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-29T04:05:48.643 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-29T04:05:48.708 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-29T04:05:48.716 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-29T04:05:48.763 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-29T04:05:48.770 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-29T04:05:48.808 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-29T04:05:48.849 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:05:48.873 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-29T04:05:48.906 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:05:48.908 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-29T04:05:48.952 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-29T04:05:48.985 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-29T04:05:49.012 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-29T04:05:49.081 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-29T04:05:49.125 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9. 39/198 2024-03-29T04:05:49.128 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:05:49.169 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el 40/198 2024-03-29T04:05:49.197 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_6 41/198 2024-03-29T04:05:49.236 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-29T04:05:49.265 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-29T04:05:49.311 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-29T04:05:49.332 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-29T04:05:49.361 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-29T04:05:49.385 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-29T04:05:49.388 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-29T04:05:49.432 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-29T04:05:49.506 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-29T04:05:49.518 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-29T04:05:49.564 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-29T04:05:49.568 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-29T04:05:49.606 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-29T04:05:49.688 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-29T04:05:49.739 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-29T04:05:49.782 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-29T04:05:49.815 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-29T04:05:49.842 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-29T04:05:49.913 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-29T04:05:49.948 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9. 39/198 2024-03-29T04:05:49.992 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el 40/198 2024-03-29T04:05:50.020 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_6 41/198 2024-03-29T04:05:50.089 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-29T04:05:50.094 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-29T04:05:50.134 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-29T04:05:50.168 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-29T04:05:50.208 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-29T04:05:50.247 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-29T04:05:50.304 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-29T04:05:50.313 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-29T04:05:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-29T04:05:50.353 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-29T04:05:50.395 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-29T04:05:50.402 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-29T04:05:50.439 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_6 53/198 2024-03-29T04:05:50.470 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-29T04:05:50.747 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-29T04:05:50.921 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-29T04:05:50.932 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-29T04:05:51.000 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-29T04:05:51.029 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-29T04:05:51.078 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-29T04:05:51.081 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-29T04:05:51.119 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-29T04:05:51.128 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-29T04:05:51.174 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-29T04:05:51.227 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-29T04:05:51.242 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-29T04:05:51.262 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_6 53/198 2024-03-29T04:05:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-29T04:05:51.293 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-29T04:05:51.348 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-29T04:05:51.421 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-29T04:05:51.451 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-29T04:05:51.488 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-29T04:05:51.543 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-29T04:05:51.601 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-29T04:05:51.601 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-29T04:05:51.622 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-29T04:05:51.664 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-29T04:05:51.798 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-29T04:05:51.831 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-29T04:05:51.836 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-29T04:05:51.866 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-29T04:05:51.911 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-29T04:05:51.933 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-29T04:05:51.956 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-29T04:05:52.027 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-29T04:05:52.074 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-29T04:05:52.109 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-29T04:05:52.115 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-29T04:05:52.141 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-29T04:05:52.158 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-29T04:05:52.165 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-29T04:05:52.213 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-29T04:05:52.257 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-29T04:05:52.286 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-29T04:05:52.318 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-29T04:05:52.346 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-29T04:05:52.382 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-29T04:05:52.392 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-29T04:05:52.428 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-29T04:05:52.471 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-29T04:05:52.481 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-29T04:05:52.522 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-29T04:05:52.539 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-29T04:05:52.588 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-29T04:05:52.644 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-29T04:05:52.656 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-29T04:05:52.711 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-29T04:05:52.741 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-29T04:05:52.783 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-29T04:05:52.827 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-29T04:05:52.869 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-29T04:05:52.876 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-29T04:05:52.916 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-29T04:05:52.957 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-29T04:05:52.997 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-29T04:05:53.007 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-29T04:05:53.031 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-29T04:05:53.034 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-29T04:05:53.065 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-29T04:05:53.096 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-29T04:05:53.114 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-29T04:05:53.124 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-29T04:05:53.158 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-29T04:05:53.197 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-29T04:05:53.229 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-29T04:05:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-29T04:05:53.259 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-29T04:05:53.334 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-29T04:05:53.369 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-29T04:05:53.388 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-29T04:05:53.403 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-29T04:05:53.444 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-29T04:05:53.479 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-29T04:05:53.568 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-29T04:05:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-29T04:05:53.649 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-29T04:05:53.701 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-29T04:05:53.732 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-29T04:05:53.766 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-29T04:05:53.779 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-29T04:05:53.839 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-29T04:05:53.877 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-29T04:05:53.925 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-29T04:05:54.033 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-29T04:05:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-29T04:05:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-29T04:05:54.260 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-29T04:05:54.293 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-29T04:05:54.322 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-29T04:05:54.350 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-29T04:05:54.385 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-29T04:05:54.424 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-29T04:05:54.456 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-29T04:05:54.486 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-29T04:05:54.562 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-29T04:05:54.596 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-29T04:05:54.630 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-29T04:05:54.656 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-29T04:05:54.679 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-29T04:05:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-29T04:05:54.714 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-29T04:05:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-29T04:05:54.796 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-29T04:05:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-29T04:05:54.883 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-29T04:05:54.884 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-29T04:05:54.923 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-29T04:05:54.936 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-29T04:05:54.968 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-29T04:05:55.015 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-29T04:05:55.101 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-29T04:05:55.112 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-29T04:05:55.137 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-29T04:05:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-29T04:05:55.215 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-29T04:05:55.859 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-29T04:05:55.897 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-29T04:05:55.942 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-29T04:05:55.970 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-29T04:05:55.976 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-29T04:05:56.024 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-29T04:05:56.112 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-29T04:05:56.160 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-29T04:05:56.336 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-29T04:05:56.366 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-29T04:05:56.443 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-29T04:05:57.110 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-29T04:05:57.192 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-29T04:05:57.219 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-29T04:05:57.280 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-29T04:05:57.310 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-29T04:05:57.341 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-29T04:05:57.369 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-29T04:05:57.682 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-29T04:05:57.707 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 124/198 2024-03-29T04:05:57.797 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 124/198 2024-03-29T04:05:57.829 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 125/198 2024-03-29T04:05:57.864 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-29T04:05:58.325 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-29T04:05:58.360 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 128/198 2024-03-29T04:05:58.431 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 128/198 2024-03-29T04:05:58.453 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 129/198 2024-03-29T04:05:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-29T04:05:58.555 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-29T04:05:58.586 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-29T04:05:58.617 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-29T04:05:58.645 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-29T04:05:58.957 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-29T04:05:58.983 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 124/198 2024-03-29T04:05:59.073 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 124/198 2024-03-29T04:05:59.104 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 125/198 2024-03-29T04:05:59.140 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-29T04:05:59.604 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-29T04:05:59.628 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 128/198 2024-03-29T04:05:59.682 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 128/198 2024-03-29T04:05:59.705 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 129/198 2024-03-29T04:06:00.676 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:06:00.708 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:06:01.260 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:06:01.303 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 131/198 2024-03-29T04:06:01.399 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 131/198 2024-03-29T04:06:01.399 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-29T04:06:01.399 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-29T04:06:01.399 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:01.429 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 132/198 2024-03-29T04:06:01.952 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:06:01.984 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:06:02.535 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:06:02.571 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 131/198 2024-03-29T04:06:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 131/198 2024-03-29T04:06:02.663 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-29T04:06:02.663 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-29T04:06:02.663 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:02.698 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 132/198 2024-03-29T04:06:13.419 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 132/198 2024-03-29T04:06:13.419 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /sys 2024-03-29T04:06:13.419 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /proc 2024-03-29T04:06:13.419 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /mnt 2024-03-29T04:06:13.419 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /var/tmp 2024-03-29T04:06:13.419 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /home 2024-03-29T04:06:13.420 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /root 2024-03-29T04:06:13.420 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /tmp 2024-03-29T04:06:13.420 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:13.600 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 133/198 2024-03-29T04:06:13.618 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 133/198 2024-03-29T04:06:13.618 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:13.618 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-29T04:06:13.618 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 132/198 2024-03-29T04:06:14.107 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-03-29T04:06:14.107 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-03-29T04:06:14.107 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-03-29T04:06:14.107 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-03-29T04:06:14.107 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-03-29T04:06:14.107 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-03-29T04:06:14.107 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-03-29T04:06:14.107 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:14.271 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 133/198 2024-03-29T04:06:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 133/198 2024-03-29T04:06:14.290 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:14.290 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-29T04:06:14.290 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:15.103 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 134/198 2024-03-29T04:06:15.129 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 134/198 2024-03-29T04:06:15.129 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:15.129 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-29T04:06:15.129 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:06:15.129 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:06:15.129 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:15.354 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 135/198 2024-03-29T04:06:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 135/198 2024-03-29T04:06:15.377 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:15.377 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-29T04:06:15.377 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:06:15.377 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:06:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:15.749 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 134/198 2024-03-29T04:06:15.775 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 134/198 2024-03-29T04:06:15.775 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:15.775 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-29T04:06:15.775 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:06:15.775 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:06:15.775 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:15.812 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 136/198 2024-03-29T04:06:15.835 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 136/198 2024-03-29T04:06:15.835 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:15.835 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-29T04:06:15.835 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:06:15.835 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:06:15.835 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:15.922 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-29T04:06:15.991 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 135/198 2024-03-29T04:06:16.014 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 135/198 2024-03-29T04:06:16.014 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:16.014 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-29T04:06:16.014 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:06:16.015 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:06:16.015 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:16.383 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 138/198 2024-03-29T04:06:16.415 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 138/198 2024-03-29T04:06:16.450 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 136/198 2024-03-29T04:06:16.453 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.e 139/198 2024-03-29T04:06:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 136/198 2024-03-29T04:06:16.472 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:16.472 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-29T04:06:16.472 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:06:16.473 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:06:16.473 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:16.513 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73. 140/198 2024-03-29T04:06:16.544 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 141/198 2024-03-29T04:06:16.561 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-29T04:06:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 138/198 2024-03-29T04:06:17.051 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 138/198 2024-03-29T04:06:17.083 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.e 139/198 2024-03-29T04:06:17.144 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73. 140/198 2024-03-29T04:06:17.174 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 141/198 2024-03-29T04:06:17.578 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 141/198 2024-03-29T04:06:17.620 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 142/198 2024-03-29T04:06:18.214 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 141/198 2024-03-29T04:06:18.253 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 142/198 2024-03-29T04:06:18.579 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 142/198 2024-03-29T04:06:18.612 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 143/198 2024-03-29T04:06:18.714 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 143/198 2024-03-29T04:06:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el 144/198 2024-03-29T04:06:18.906 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 145/198 2024-03-29T04:06:18.929 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 145/198 2024-03-29T04:06:18.929 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:18.929 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-29T04:06:18.929 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:06:18.929 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:06:18.929 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:18.973 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 146/198 2024-03-29T04:06:18.984 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 146/198 2024-03-29T04:06:19.036 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 147/198 2024-03-29T04:06:19.211 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 142/198 2024-03-29T04:06:19.243 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 143/198 2024-03-29T04:06:19.343 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 143/198 2024-03-29T04:06:19.531 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el 144/198 2024-03-29T04:06:19.562 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 145/198 2024-03-29T04:06:19.584 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 145/198 2024-03-29T04:06:19.584 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:19.585 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-29T04:06:19.585 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:06:19.585 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:06:19.585 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:19.629 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 146/198 2024-03-29T04:06:19.640 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 146/198 2024-03-29T04:06:19.688 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 147/198 2024-03-29T04:06:19.928 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-29T04:06:19.958 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 149/198 2024-03-29T04:06:19.982 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 149/198 2024-03-29T04:06:19.982 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:19.982 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-29T04:06:19.982 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:06:19.982 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:06:19.982 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:20.026 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 150/198 2024-03-29T04:06:20.048 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 150/198 2024-03-29T04:06:20.048 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:20.048 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-29T04:06:20.048 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:20.093 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:06:20.116 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:06:20.116 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:20.116 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-29T04:06:20.116 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:06:20.116 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:06:20.116 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:20.435 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 152/198 2024-03-29T04:06:20.458 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 152/198 2024-03-29T04:06:20.458 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:20.458 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-29T04:06:20.458 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:06:20.458 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:06:20.458 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:20.568 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-29T04:06:20.607 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 149/198 2024-03-29T04:06:20.629 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 149/198 2024-03-29T04:06:20.629 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:20.629 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-29T04:06:20.629 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:06:20.629 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:06:20.629 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:20.674 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 150/198 2024-03-29T04:06:20.695 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 150/198 2024-03-29T04:06:20.695 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:20.695 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-29T04:06:20.696 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:20.741 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:06:20.763 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:06:20.763 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:20.763 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-29T04:06:20.764 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:06:20.764 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:06:20.764 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:21.084 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 152/198 2024-03-29T04:06:21.106 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 152/198 2024-03-29T04:06:21.106 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:06:21.106 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-29T04:06:21.106 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:06:21.106 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:06:21.107 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:24.981 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 153/198 2024-03-29T04:06:25.023 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 154/198 2024-03-29T04:06:25.053 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 155/198 2024-03-29T04:06:25.080 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-29T04:06:25.271 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 157/198 2024-03-29T04:06:25.380 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-29T04:06:25.420 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 159/198 2024-03-29T04:06:25.472 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-29T04:06:25.528 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-29T04:06:25.563 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-29T04:06:25.596 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_ 163/198 2024-03-29T04:06:25.597 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 153/198 2024-03-29T04:06:25.628 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-29T04:06:25.638 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 154/198 2024-03-29T04:06:25.657 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-29T04:06:25.668 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 155/198 2024-03-29T04:06:25.695 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-29T04:06:25.721 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-29T04:06:25.784 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-29T04:06:25.823 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-29T04:06:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 157/198 2024-03-29T04:06:26.004 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-29T04:06:26.044 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 159/198 2024-03-29T04:06:26.097 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-29T04:06:26.152 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-29T04:06:26.196 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-29T04:06:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_ 163/198 2024-03-29T04:06:26.268 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-29T04:06:26.298 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-29T04:06:26.366 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-29T04:06:26.423 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-29T04:06:26.438 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:06:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-29T04:06:26.478 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:06:26.498 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:06:27.078 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:06:27.114 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:06:27.134 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:06:27.331 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 170/198 2024-03-29T04:06:27.401 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-29T04:06:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-29T04:06:27.537 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-29T04:06:27.571 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-29T04:06:27.596 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-29T04:06:27.621 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-29T04:06:27.648 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-29T04:06:27.648 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-29T04:06:27.736 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-29T04:06:27.774 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-29T04:06:27.774 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-29T04:06:27.858 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-29T04:06:27.885 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-29T04:06:27.918 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-29T04:06:27.943 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-29T04:06:27.952 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 170/198 2024-03-29T04:06:27.968 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-29T04:06:28.003 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-29T04:06:28.034 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-29T04:06:28.049 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-29T04:06:28.075 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-29T04:06:28.081 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-29T04:06:28.098 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-29T04:06:28.123 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-29T04:06:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-29T04:06:28.164 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-29T04:06:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-29T04:06:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:06:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:06:28.196 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-29T04:06:28.230 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-29T04:06:28.255 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-29T04:06:28.282 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-29T04:06:28.282 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-29T04:06:28.368 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:06:28.369 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-29T04:06:28.378 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-29T04:06:28.408 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-29T04:06:28.408 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-29T04:06:28.443 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-29T04:06:28.468 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-29T04:06:28.492 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-29T04:06:28.499 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-29T04:06:28.518 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-29T04:06:28.547 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-29T04:06:28.552 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-29T04:06:28.577 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-29T04:06:28.577 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-29T04:06:28.602 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-29T04:06:28.637 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-29T04:06:28.642 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-29T04:06:28.675 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-29T04:06:28.707 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-29T04:06:28.732 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-29T04:06:28.757 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-29T04:06:28.783 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-29T04:06:28.808 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-29T04:06:28.815 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:06:28.815 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:06:29.003 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:06:29.003 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-29T04:06:29.086 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-29T04:06:29.113 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-29T04:06:29.143 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-29T04:06:29.170 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-29T04:06:29.203 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-29T04:06:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-29T04:06:33.996 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 1/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 2/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 3/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 4/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 5/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 6/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 7/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 8/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 9/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 12/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 13/198 2024-03-29T04:06:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 14/198 2024-03-29T04:06:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_ 15/198 2024-03-29T04:06:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:06:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 17/198 2024-03-29T04:06:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_6 18/198 2024-03-29T04:06:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 19/198 2024-03-29T04:06:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 20/198 2024-03-29T04:06:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el 21/198 2024-03-29T04:06:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9. 22/198 2024-03-29T04:06:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_6 23/198 2024-03-29T04:06:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 24/198 2024-03-29T04:06:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 25/198 2024-03-29T04:06:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 26/198 2024-03-29T04:06:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 27/198 2024-03-29T04:06:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 28/198 2024-03-29T04:06:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 29/198 2024-03-29T04:06:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73. 30/198 2024-03-29T04:06:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 31/198 2024-03-29T04:06:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 32/198 2024-03-29T04:06:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 33/198 2024-03-29T04:06:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el 34/198 2024-03-29T04:06:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 35/198 2024-03-29T04:06:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.e 36/198 2024-03-29T04:06:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 37/198 2024-03-29T04:06:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 38/198 2024-03-29T04:06:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 39/198 2024-03-29T04:06:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-29T04:06:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-29T04:06:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-29T04:06:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-29T04:06:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-29T04:06:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-29T04:06:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-29T04:06:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-29T04:06:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-29T04:06:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-29T04:06:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-29T04:06:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-29T04:06:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-29T04:06:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-29T04:06:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-29T04:06:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-29T04:06:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-29T04:06:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-29T04:06:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-29T04:06:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-29T04:06:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-29T04:06:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-29T04:06:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-29T04:06:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-29T04:06:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-29T04:06:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-29T04:06:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-29T04:06:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-29T04:06:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-29T04:06:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-03-29T04:06:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-03-29T04:06:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-03-29T04:06:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-03-29T04:06:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-03-29T04:06:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-03-29T04:06:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-03-29T04:06:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-29T04:06:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-03-29T04:06:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-03-29T04:06:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-03-29T04:06:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-03-29T04:06:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-03-29T04:06:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-03-29T04:06:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-03-29T04:06:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-03-29T04:06:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-03-29T04:06:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-03-29T04:06:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-03-29T04:06:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-03-29T04:06:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-03-29T04:06:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-03-29T04:06:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-29T04:06:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-29T04:06:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-29T04:06:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-29T04:06:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-29T04:06:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-29T04:06:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-29T04:06:34.029 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-29T04:06:34.029 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-29T04:06:34.029 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-29T04:06:34.029 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-29T04:06:34.029 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-29T04:06:34.029 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-29T04:06:34.029 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-29T04:06:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-29T04:06:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-29T04:06:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-29T04:06:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-29T04:06:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-29T04:06:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-29T04:06:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-29T04:06:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-29T04:06:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-29T04:06:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-29T04:06:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-29T04:06:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-29T04:06:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-29T04:06:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-29T04:06:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-29T04:06:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-29T04:06:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-29T04:06:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-29T04:06:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-29T04:06:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-29T04:06:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-29T04:06:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-29T04:06:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-29T04:06:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-29T04:06:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-29T04:06:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-29T04:06:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-29T04:06:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-29T04:06:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-29T04:06:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-29T04:06:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-29T04:06:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-29T04:06:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-29T04:06:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:06:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-29T04:06:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 153/198 2024-03-29T04:06:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-29T04:06:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-03-29T04:06:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-29T04:06:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-03-29T04:06:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-29T04:06:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-03-29T04:06:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-29T04:06:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-03-29T04:06:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-29T04:06:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-03-29T04:06:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-03-29T04:06:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-29T04:06:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-29T04:06:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-29T04:06:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-29T04:06:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-29T04:06:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-29T04:06:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-29T04:06:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-29T04:06:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-29T04:06:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-29T04:06:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-29T04:06:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-29T04:06:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-29T04:06:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-29T04:06:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-29T04:06:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-29T04:06:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-29T04:06:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-29T04:06:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-29T04:06:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-29T04:06:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-29T04:06:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-29T04:06:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-29T04:06:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-29T04:06:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-29T04:06:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-29T04:06:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-29T04:06:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-29T04:06:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-29T04:06:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-29T04:06:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-29T04:06:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-29T04:06:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-29T04:06:34.661 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-29T04:06:34.661 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 1/198 2024-03-29T04:06:34.661 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 2/198 2024-03-29T04:06:34.661 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 3/198 2024-03-29T04:06:34.661 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 4/198 2024-03-29T04:06:34.661 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 5/198 2024-03-29T04:06:34.661 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 6/198 2024-03-29T04:06:34.661 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 7/198 2024-03-29T04:06:34.661 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 8/198 2024-03-29T04:06:34.662 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 9/198 2024-03-29T04:06:34.662 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:06:34.662 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:06:34.662 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 12/198 2024-03-29T04:06:34.662 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 13/198 2024-03-29T04:06:34.662 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 14/198 2024-03-29T04:06:34.662 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_ 15/198 2024-03-29T04:06:34.662 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:06:34.662 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 17/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_6 18/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 19/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 20/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el 21/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9. 22/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_6 23/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 24/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 25/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 26/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 27/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 28/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 29/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73. 30/198 2024-03-29T04:06:34.663 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 31/198 2024-03-29T04:06:34.664 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 32/198 2024-03-29T04:06:34.664 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 33/198 2024-03-29T04:06:34.664 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el 34/198 2024-03-29T04:06:34.664 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 35/198 2024-03-29T04:06:34.664 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.e 36/198 2024-03-29T04:06:34.664 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 37/198 2024-03-29T04:06:34.664 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 38/198 2024-03-29T04:06:34.664 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 39/198 2024-03-29T04:06:34.664 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-29T04:06:34.664 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-29T04:06:34.664 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-29T04:06:34.664 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-29T04:06:34.664 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-03-29T04:06:34.665 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-03-29T04:06:34.666 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-03-29T04:06:34.666 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-03-29T04:06:34.666 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-29T04:06:34.666 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-29T04:06:34.666 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-29T04:06:34.666 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-29T04:06:34.666 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-29T04:06:34.666 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-29T04:06:34.666 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-29T04:06:34.666 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-29T04:06:34.666 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-29T04:06:34.666 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-29T04:06:34.666 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-03-29T04:06:34.667 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-03-29T04:06:34.668 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-03-29T04:06:34.668 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-29T04:06:34.668 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-03-29T04:06:34.668 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-03-29T04:06:34.668 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-03-29T04:06:34.668 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-03-29T04:06:34.668 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-03-29T04:06:34.668 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-03-29T04:06:34.668 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-03-29T04:06:34.668 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-03-29T04:06:34.668 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-03-29T04:06:34.669 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-03-29T04:06:34.669 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-03-29T04:06:34.669 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-03-29T04:06:34.669 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-03-29T04:06:34.669 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-03-29T04:06:34.669 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-29T04:06:34.669 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-29T04:06:34.670 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-29T04:06:34.670 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-29T04:06:34.670 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-29T04:06:34.670 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-29T04:06:34.670 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-29T04:06:34.670 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-29T04:06:34.670 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-29T04:06:34.670 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-29T04:06:34.670 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-29T04:06:34.670 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-29T04:06:34.670 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-29T04:06:34.670 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-29T04:06:34.671 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-29T04:06:34.672 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 153/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-03-29T04:06:34.673 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-29T04:06:34.674 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-29T04:06:34.675 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-29T04:06:34.675 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-29T04:06:34.675 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-29T04:06:34.675 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-29T04:06:34.675 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-29T04:06:34.675 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-29T04:06:34.675 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-29T04:06:34.675 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-29T04:06:34.675 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-29T04:06:34.675 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-29T04:06:34.675 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-29T04:06:34.676 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-29T04:06:34.676 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-29T04:06:34.676 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-29T04:06:34.676 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-29T04:06:34.676 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-29T04:06:34.676 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-29T04:06:34.676 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-29T04:06:34.676 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-29T04:06:34.676 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-29T04:06:34.676 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-29T04:06:34.676 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-29T04:06:34.676 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-29T04:06:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-29T04:06:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:36.944 INFO:teuthology.orchestra.run.smithi154.stdout:Upgraded: 2024-03-29T04:06:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-29T04:06:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-29T04:06:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-29T04:06:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-29T04:06:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: acl-2.3.1-4.el9.x86_64 2024-03-29T04:06:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-29T04:06:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-29T04:06:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-29T04:06:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-29T04:06:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-29T04:06:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-29T04:06:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-29T04:06:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-29T04:06:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-29T04:06:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-29T04:06:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-29T04:06:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-29T04:06:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-29T04:06:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-29T04:06:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-29T04:06:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-29T04:06:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-29T04:06:36.946 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-03-29T04:06:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-29T04:06:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: automake-1.16.2-8.el9.noarch 2024-03-29T04:06:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: bison-3.7.4-5.el9.x86_64 2024-03-29T04:06:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-29T04:06:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-29T04:06:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: flex-2.6.4-9.el9.x86_64 2024-03-29T04:06:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-29T04:06:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-29T04:06:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-29T04:06:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-29T04:06:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-29T04:06:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-29T04:06:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: jq-1.6-16.el9.x86_64 2024-03-29T04:06:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-29T04:06:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-29T04:06:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-29T04:06:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-29T04:06:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: lua-5.4.4-4.el9.x86_64 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-13.14-1.el9.x86_64 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-29T04:06:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-29T04:06:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-29T04:06:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply-3.11-14.el9.noarch 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-29T04:06:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-29T04:06:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.957 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:36.957 INFO:teuthology.orchestra.run.smithi154.stdout: re2-1:20211101-3.el9.x86_64 2024-03-29T04:06:36.957 INFO:teuthology.orchestra.run.smithi154.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-29T04:06:36.957 INFO:teuthology.orchestra.run.smithi154.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-29T04:06:36.957 INFO:teuthology.orchestra.run.smithi154.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-29T04:06:36.957 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-29T04:06:36.957 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-29T04:06:36.957 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-29T04:06:36.958 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:36.958 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-03-29T04:06:37.429 DEBUG:teuthology.parallel:result is None 2024-03-29T04:06:37.927 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-29T04:06:37.927 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:37.928 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-03-29T04:06:37.928 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-29T04:06:37.928 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-29T04:06:37.928 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-29T04:06:37.928 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-29T04:06:37.928 INFO:teuthology.orchestra.run.smithi097.stdout: acl-2.3.1-4.el9.x86_64 2024-03-29T04:06:37.928 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-29T04:06:37.928 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-29T04:06:37.928 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-29T04:06:37.928 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-29T04:06:37.928 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-29T04:06:37.928 INFO:teuthology.orchestra.run.smithi097.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-29T04:06:37.928 INFO:teuthology.orchestra.run.smithi097.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-03-29T04:06:37.929 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-29T04:06:37.930 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-03-29T04:06:37.930 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-03-29T04:06:37.930 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-29T04:06:37.930 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.930 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.930 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.930 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.930 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:37.930 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.930 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.930 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.930 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:37.930 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.931 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: jq-1.6-16.el9.x86_64 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.932 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.933 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-29T04:06:37.933 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-29T04:06:37.933 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-29T04:06:37.933 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-29T04:06:37.933 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-29T04:06:37.933 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.933 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.933 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-29T04:06:37.933 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.933 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-29T04:06:37.933 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-29T04:06:37.933 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-29T04:06:37.933 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.14-1.el9.x86_64 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-29T04:06:37.935 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-29T04:06:37.935 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-29T04:06:37.935 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-29T04:06:37.935 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-29T04:06:37.935 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-29T04:06:37.935 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-29T04:06:37.935 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.935 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.935 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.935 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-29T04:06:37.935 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-29T04:06:37.935 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-29T04:06:37.935 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-29T04:06:37.936 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-03-29T04:06:37.937 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-29T04:06:37.938 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-29T04:06:37.938 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-29T04:06:37.938 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-29T04:06:37.938 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-29T04:06:37.938 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-29T04:06:37.938 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-29T04:06:37.938 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-29T04:06:37.938 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-29T04:06:37.938 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.938 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.938 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-29T04:06:37.938 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-29T04:06:37.938 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-29T04:06:37.939 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-29T04:06:37.940 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-29T04:06:37.940 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.940 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.940 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:06:37.940 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-03-29T04:06:37.940 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-29T04:06:37.940 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-29T04:06:37.940 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-29T04:06:37.940 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-29T04:06:37.940 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-29T04:06:37.941 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-29T04:06:37.941 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:37.941 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-03-29T04:06:38.461 DEBUG:teuthology.parallel:result is None 2024-03-29T04:06:38.462 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:06:38.602 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-29T04:06:38.624 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-2520.g69e2cd73.el9 2024-03-29T04:06:38.624 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2520.g69e2cd73.el9 2024-03-29T04:06:38.625 INFO:teuthology.task.install:The correct ceph version 19.0.0-2520.g69e2cd73 is installed. 2024-03-29T04:06:38.627 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:06:38.775 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-29T04:06:38.801 INFO:teuthology.orchestra.run.smithi154.stdout:19.0.0-2520.g69e2cd73.el9 2024-03-29T04:06:38.801 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2520.g69e2cd73.el9 2024-03-29T04:06:38.801 INFO:teuthology.task.install:The correct ceph version 19.0.0-2520.g69e2cd73 is installed. 2024-03-29T04:06:38.803 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-29T04:06:38.803 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:06:38.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T04:06:38.834 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:06:38.834 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T04:06:38.876 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-29T04:06:38.876 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:06:38.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T04:06:38.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T04:06:38.971 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:06:38.971 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T04:06:39.001 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T04:06:39.072 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-29T04:06:39.072 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:06:39.072 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T04:06:39.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T04:06:39.167 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:06:39.167 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T04:06:39.196 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T04:06:39.265 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-29T04:06:39.265 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:06:39.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T04:06:39.292 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T04:06:39.358 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:06:39.358 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T04:06:39.386 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T04:06:39.454 INFO:teuthology.run_tasks:Running task ceph... 2024-03-29T04:06:39.535 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-29T04:06:39.535 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /var/log/ceph 2024-03-29T04:06:39.538 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /var/log/ceph 2024-03-29T04:06:39.565 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-29T04:06:39.565 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-29T04:06:39.605 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-29T04:06:39.632 INFO:tasks.ceph:Creating extra log directories... 2024-03-29T04:06:39.632 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-29T04:06:39.672 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-29T04:06:39.706 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-29T04:06:39.706 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-29T04:06:39.706 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-29T04:06:39.707 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:06:39.707 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-29T04:06:39.751 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-29T04:06:39.870 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-29T04:06:39.897 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:06:39.897 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-29T04:06:39.911 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-29T04:06:40.031 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-29T04:06:40.064 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-29T04:06:40.064 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-29T04:06:40.065 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076', 'branch': 'wip-mchangir-testing1-main-20240328-072412', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7629076', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-testing1-main-20240328-072412'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-testing1-main-20240328-072412', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8207, 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-mchangir-testing1-main-20240328-072412', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi097.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIZEzUaPO6wk7oZ1TFA+ugiuBrfnoH0JnaSZXeUIPE1MO0Hym+6XkM68t6vlyVE4ughsiaFekDbZCWB3/gc3H/Y=', 'smithi154.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEWPhAtYviuzhJO7VkwnqL3RDKbYNrsCin0sYC7fKMpz9KQzEDIwmoNQsE1m1jOSCNOscybOnMO0aANEZJoPBJs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_uninlining']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-29_02:01:30', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-29T04:06:40.065 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-29T04:06:40.067 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-29T04:06:40.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/run/ceph 2024-03-29T04:06:40.122 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/run/ceph 2024-03-29T04:06:40.180 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:06:40.180 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T04:06:40.202 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T04:06:40.203 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-03-29T04:06:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T04:06:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:06:40.257 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-29T04:06:40.257 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:06:40.257 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:06:40.257 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-29 04:06:29.659543918 +0000 2024-03-29T04:06:40.258 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-29 04:04:13.912468730 +0000 2024-03-29T04:06:40.258 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-29 04:04:13.912468730 +0000 2024-03-29T04:06:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-29 04:04:13.912468730 +0000 2024-03-29T04:06:40.258 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T04:06:40.322 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-29T04:06:40.323 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-29T04:06:40.323 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000192341 s, 2.7 MB/s 2024-03-29T04:06:40.325 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T04:06:40.380 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-03-29T04:06:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T04:06:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:06:40.435 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-29T04:06:40.435 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:06:40.435 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:06:40.435 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-29 04:06:29.659543918 +0000 2024-03-29T04:06:40.435 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-29 04:04:14.273462570 +0000 2024-03-29T04:06:40.435 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-29 04:04:14.273462570 +0000 2024-03-29T04:06:40.435 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-29 04:04:14.273462570 +0000 2024-03-29T04:06:40.436 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T04:06:40.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-29T04:06:40.501 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-29T04:06:40.502 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000184443 s, 2.8 MB/s 2024-03-29T04:06:40.502 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T04:06:40.558 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-03-29T04:06:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T04:06:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:06:40.613 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-29T04:06:40.613 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:06:40.613 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:06:40.613 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-29 04:06:29.660543901 +0000 2024-03-29T04:06:40.613 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-29 04:04:14.607456871 +0000 2024-03-29T04:06:40.613 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-29 04:04:14.607456871 +0000 2024-03-29T04:06:40.613 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-29 04:04:14.607456871 +0000 2024-03-29T04:06:40.614 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T04:06:40.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-29T04:06:40.679 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-29T04:06:40.679 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000217701 s, 2.4 MB/s 2024-03-29T04:06:40.680 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T04:06:40.736 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-03-29T04:06:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T04:06:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:06:40.792 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-29T04:06:40.792 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:06:40.792 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:06:40.792 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-03-29 04:06:29.660543901 +0000 2024-03-29T04:06:40.792 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-03-29 04:04:14.959450864 +0000 2024-03-29T04:06:40.792 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-03-29 04:04:14.959450864 +0000 2024-03-29T04:06:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-03-29 04:04:14.959450864 +0000 2024-03-29T04:06:40.793 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T04:06:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-03-29T04:06:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-03-29T04:06:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000235069 s, 2.2 MB/s 2024-03-29T04:06:40.858 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T04:06:40.914 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:06:40.914 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:06:40.914 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T04:06:40.930 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T04:06:40.930 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2024-03-29T04:06:40.985 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T04:06:40.986 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:06:40.986 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-29T04:06:40.986 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:06:40.986 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:06:40.986 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-03-29 04:06:28.998354515 +0000 2024-03-29T04:06:40.986 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-03-29 04:04:15.807896413 +0000 2024-03-29T04:06:40.986 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-03-29 04:04:15.807896413 +0000 2024-03-29T04:06:40.986 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-03-29 04:04:15.807896413 +0000 2024-03-29T04:06:40.986 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T04:06:41.052 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-03-29T04:06:41.053 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-03-29T04:06:41.053 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000246578 s, 2.1 MB/s 2024-03-29T04:06:41.054 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T04:06:41.109 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2024-03-29T04:06:41.164 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T04:06:41.164 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:06:41.164 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-29T04:06:41.164 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:06:41.165 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:06:41.165 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-03-29 04:06:28.998354515 +0000 2024-03-29T04:06:41.165 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-03-29 04:04:16.142887302 +0000 2024-03-29T04:06:41.165 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-03-29 04:04:16.142887302 +0000 2024-03-29T04:06:41.165 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-03-29 04:04:16.142887302 +0000 2024-03-29T04:06:41.165 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T04:06:41.229 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-03-29T04:06:41.230 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-03-29T04:06:41.230 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000195103 s, 2.6 MB/s 2024-03-29T04:06:41.231 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T04:06:41.287 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2024-03-29T04:06:41.341 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T04:06:41.341 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:06:41.341 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-29T04:06:41.342 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:06:41.342 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:06:41.342 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-03-29 04:06:28.998354515 +0000 2024-03-29T04:06:41.342 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-03-29 04:04:16.482878054 +0000 2024-03-29T04:06:41.342 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-03-29 04:04:16.482878054 +0000 2024-03-29T04:06:41.342 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-03-29 04:04:16.482878054 +0000 2024-03-29T04:06:41.342 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T04:06:41.407 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-03-29T04:06:41.407 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-03-29T04:06:41.407 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000227834 s, 2.2 MB/s 2024-03-29T04:06:41.409 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T04:06:41.464 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2024-03-29T04:06:41.518 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T04:06:41.518 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:06:41.518 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-29T04:06:41.519 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:06:41.519 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:06:41.519 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-03-29 04:06:28.999354489 +0000 2024-03-29T04:06:41.519 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-03-29 04:04:16.825868726 +0000 2024-03-29T04:06:41.519 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-03-29 04:04:16.825868726 +0000 2024-03-29T04:06:41.519 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-03-29 04:04:16.825868726 +0000 2024-03-29T04:06:41.519 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T04:06:41.584 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-03-29T04:06:41.584 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-03-29T04:06:41.584 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000206032 s, 2.5 MB/s 2024-03-29T04:06:41.585 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T04:06:41.641 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:06:41.641 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-29T04:06:41.641 INFO:tasks.ceph:Generating config... 2024-03-29T04:06:41.642 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-29T04:06:41.642 INFO:tasks.ceph:[client] debug client = 20 2024-03-29T04:06:41.642 INFO:tasks.ceph:[client] debug ms = 1 2024-03-29T04:06:41.642 INFO:tasks.ceph:[client] pid file = 2024-03-29T04:06:41.643 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-29T04:06:41.643 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-29T04:06:41.643 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-29T04:06:41.643 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-29T04:06:41.643 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-29T04:06:41.643 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-29T04:06:41.643 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-29T04:06:41.643 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-29T04:06:41.643 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-29T04:06:41.643 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-29T04:06:41.643 INFO:tasks.ceph:[mds] mds_bal_replicate_threshold = 1 2024-03-29T04:06:41.643 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-29T04:06:41.643 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-29T04:06:41.643 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-29T04:06:41.643 INFO:tasks.ceph:[mgr] debug client = 20 2024-03-29T04:06:41.643 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-03-29T04:06:41.644 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-29T04:06:41.644 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-29T04:06:41.644 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-29T04:06:41.644 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-29T04:06:41.644 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-29T04:06:41.644 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-03-29T04:06:41.644 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-29T04:06:41.644 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-29T04:06:41.644 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-29T04:06:41.644 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-29T04:06:41.644 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-29T04:06:41.644 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-29T04:06:41.644 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-29T04:06:41.644 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-29T04:06:41.644 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-29T04:06:41.644 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-29T04:06:41.645 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-29T04:06:41.645 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-29T04:06:41.645 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-29T04:06:41.645 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-29T04:06:41.645 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-29T04:06:41.645 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-29T04:06:41.645 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-29T04:06:41.645 INFO:tasks.ceph:Setting up mon.a... 2024-03-29T04:06:41.645 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-29T04:06:41.694 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.keyring 2024-03-29T04:06:41.697 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-29T04:06:41.741 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T04:06:41.808 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.97'), ('mon.b', '172.21.15.154'), ('mon.c', '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]')] 2024-03-29T04:06:41.808 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.97,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-29T04:06:41.809 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:06:41.809 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-29T04:06:41.863 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.97 --add b 172.21.15.154 --addv c '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:06:41.947 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:06:41.947 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: generated fsid 51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f 2024-03-29T04:06:41.948 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = quincy 2024-03-29T04:06:41.948 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 0 2024-03-29T04:06:41.948 INFO:teuthology.orchestra.run.smithi097.stdout:fsid 51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f 2024-03-29T04:06:41.948 INFO:teuthology.orchestra.run.smithi097.stdout:last_changed 2024-03-29T04:06:41.968753+0000 2024-03-29T04:06:41.948 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-03-29T04:06:41.968753+0000 2024-03-29T04:06:41.948 INFO:teuthology.orchestra.run.smithi097.stdout:min_mon_release 17 (quincy) 2024-03-29T04:06:41.948 INFO:teuthology.orchestra.run.smithi097.stdout:election_strategy: 1 2024-03-29T04:06:41.948 INFO:teuthology.orchestra.run.smithi097.stdout:0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-03-29T04:06:41.948 INFO:teuthology.orchestra.run.smithi097.stdout:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-03-29T04:06:41.948 INFO:teuthology.orchestra.run.smithi097.stdout:2: [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] mon.c 2024-03-29T04:06:41.948 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-29T04:06:41.949 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-29T04:06:41.964 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f... 2024-03-29T04:06:41.965 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T04:06:42.007 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T04:06:42.055 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-03-29T04:06:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2024-03-29T04:06:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T04:06:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2024-03-29T04:06:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2024-03-29T04:06:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2024-03-29T04:06:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2024-03-29T04:06:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2024-03-29T04:06:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2024-03-29T04:06:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2024-03-29T04:06:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T04:06:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2024-03-29T04:06:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T04:06:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T04:06:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T04:06:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2024-03-29T04:06:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2024-03-29T04:06:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2024-03-29T04:06:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2024-03-29T04:06:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2024-03-29T04:06:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.97,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]" 2024-03-29T04:06:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on skipped message = False 2024-03-29T04:06:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = 51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f 2024-03-29T04:06:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.058 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2024-03-29T04:06:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug trim objects = true 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: bdev async discard = True 2024-03-29T04:06:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: bdev enable discard = True 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore allocator = bitmap 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2024-03-29T04:06:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: debug finisher = 20 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-29T04:06:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: mon down mkfs grace = 300 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2024-03-29T04:06:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2024-03-29T04:06:42.065 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2024-03-29T04:06:42.066 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2024-03-29T04:06:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2024-03-29T04:06:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2024-03-29T04:06:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-03-29T04:06:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2024-03-29T04:06:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2024-03-29T04:06:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2024-03-29T04:06:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2024-03-29T04:06:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: mds_bal_replicate_threshold = 1 2024-03-29T04:06:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2024-03-29T04:06:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2024-03-29T04:06:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2024-03-29T04:06:42.067 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2024-03-29T04:06:42.067 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2024-03-29T04:06:42.067 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T04:06:42.067 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2024-03-29T04:06:42.067 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.067 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2024-03-29T04:06:42.067 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.067 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2024-03-29T04:06:42.067 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2024-03-29T04:06:42.068 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.069 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.97,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]" 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on skipped message = False 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug trim objects = true 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2024-03-29T04:06:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: bdev async discard = True 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: bdev enable discard = True 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore allocator = bitmap 2024-03-29T04:06:42.072 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 1/20 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 1/20 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 4/10 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.073 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: debug finisher = 20 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2024-03-29T04:06:42.074 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T04:06:42.075 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: mon down mkfs grace = 300 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T04:06:42.076 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2024-03-29T04:06:42.077 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2024-03-29T04:06:42.078 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2024-03-29T04:06:42.078 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-03-29T04:06:42.078 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2024-03-29T04:06:42.078 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = True 2024-03-29T04:06:42.078 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2024-03-29T04:06:42.078 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = True 2024-03-29T04:06:42.078 INFO:teuthology.orchestra.run.smithi154.stdout: mds_bal_replicate_threshold = 1 2024-03-29T04:06:42.078 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2024-03-29T04:06:42.078 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2024-03-29T04:06:42.078 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2024-03-29T04:06:42.080 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-29T04:06:42.080 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-29T04:06:42.175 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-29T04:06:42.175 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:06:42.175 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-29T04:06:42.190 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:06:42.190 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-29T04:06:42.244 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T04:06:42.244 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:06:42.244 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-29T04:06:42.244 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T04:06:42.324 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:06:42.324 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:06:42.378 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T04:06:42.378 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:06:42.378 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-29T04:06:42.379 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T04:06:42.419 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:06:42.419 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:06:42.473 INFO:tasks.ceph:Setting up mon nodes... 2024-03-29T04:06:42.473 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-29T04:06:42.474 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-29T04:06:42.528 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-29T04:06:42.531 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-29T04:06:42.622 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-29T04:06:42.625 INFO:tasks.ceph:Setting up mds nodes... 2024-03-29T04:06:42.625 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-29T04:06:42.681 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-29T04:06:42.683 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-29T04:06:42.710 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-29T04:06:42.804 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-29T04:06:42.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-29T04:06:42.833 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-29T04:06:42.888 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-29T04:06:42.890 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-29T04:06:42.917 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-29T04:06:43.012 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-29T04:06:43.018 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-29T04:06:43.046 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-29T04:06:43.047 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-29T04:06:43.089 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-29T04:06:43.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-29T04:06:43.187 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-29T04:06:43.202 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-29T04:06:43.245 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-29T04:06:43.262 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-29T04:06:43.345 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-29T04:06:43.360 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-29T04:06:43.360 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-29T04:06:43.360 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-29T04:06:43.387 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:06:43.387 INFO:tasks.ceph:role: osd.0 2024-03-29T04:06:43.388 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T04:06:43.388 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-29T04:06:43.461 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:06:43.462 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:06:43.462 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:06:43.462 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:06:43.462 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:06:43.462 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:06:43.462 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:06:43.462 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:06:43.462 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:06:43.462 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:06:43.974 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-03-29T04:06:43.976 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-03-29T04:06:43.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-29T04:06:44.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-29T04:06:44.149 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-29T04:06:44.217 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:06:44.217 INFO:tasks.ceph:role: osd.1 2024-03-29T04:06:44.218 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T04:06:44.218 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-29T04:06:44.292 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:06:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:06:44.292 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:06:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:06:44.293 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:06:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:06:44.293 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:06:44.293 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:06:44.293 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:06:44.293 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:06:44.731 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-03-29T04:06:44.733 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-03-29T04:06:44.733 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-29T04:06:44.824 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-29T04:06:44.854 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-29T04:06:44.921 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:06:44.922 INFO:tasks.ceph:role: osd.2 2024-03-29T04:06:44.922 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T04:06:44.922 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-29T04:06:44.996 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:06:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:06:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:06:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:06:44.997 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:06:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:06:44.997 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:06:44.997 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:06:44.997 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:06:44.997 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:06:45.479 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-03-29T04:06:45.481 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-03-29T04:06:45.481 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-29T04:06:45.624 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-29T04:06:45.654 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-29T04:06:45.721 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:06:45.721 INFO:tasks.ceph:role: osd.3 2024-03-29T04:06:45.721 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T04:06:45.722 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-29T04:06:45.796 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:06:45.796 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:06:45.796 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:06:45.796 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:06:45.796 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:06:45.796 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:06:45.796 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:06:45.797 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:06:45.797 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:06:45.797 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:06:46.292 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-03-29T04:06:46.294 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-03-29T04:06:46.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-29T04:06:46.431 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-29T04:06:46.466 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:06:46.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:46.592+0000 7f2bbe611640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-29T04:06:46.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:46.593+0000 7f2bbe611640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-29T04:06:46.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:46.593+0000 7f2bbe611640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T04:06:46.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:46.593+0000 7f2bbe611640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T04:06:46.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:46.593+0000 7f2bbe611640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T04:06:46.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:46.593+0000 7f2bbe611640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-29T04:06:49.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-29T04:06:49.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:06:49.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:49.579+0000 7fed8f58c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-29T04:06:49.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:49.580+0000 7fed8f58c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-29T04:06:49.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:49.580+0000 7fed8f58c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T04:06:49.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:49.580+0000 7fed8f58c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T04:06:49.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:49.580+0000 7fed8f58c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T04:06:49.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:49.580+0000 7fed8f58c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-29T04:06:52.409 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-29T04:06:52.441 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:06:52.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:52.552+0000 7f38da604640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-29T04:06:52.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:52.552+0000 7f38da604640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-29T04:06:52.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:52.552+0000 7f38da604640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T04:06:52.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:52.552+0000 7f38da604640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T04:06:52.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:52.552+0000 7f38da604640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T04:06:52.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:52.553+0000 7f38da604640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-29T04:06:55.388 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-29T04:06:55.419 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:06:55.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:55.533+0000 7f495896b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-29T04:06:55.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:55.534+0000 7f495896b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-29T04:06:55.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:55.534+0000 7f495896b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T04:06:55.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:55.534+0000 7f495896b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T04:06:55.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:55.534+0000 7f495896b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T04:06:55.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:06:55.534+0000 7f495896b640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-29T04:06:58.369 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-29T04:06:58.399 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-29T04:06:58.433 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:06:58.433 INFO:tasks.ceph:role: osd.4 2024-03-29T04:06:58.433 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T04:06:58.433 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-29T04:06:58.511 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:06:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:06:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:06:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:06:58.511 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:06:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:06:58.511 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:06:58.511 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:06:58.511 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:06:58.511 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:06:59.007 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-03-29T04:06:59.007 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-03-29T04:06:59.007 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-29T04:06:59.130 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-29T04:06:59.168 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-29T04:06:59.239 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:06:59.239 INFO:tasks.ceph:role: osd.5 2024-03-29T04:06:59.239 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T04:06:59.239 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-29T04:06:59.323 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:06:59.323 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:06:59.323 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:06:59.323 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:06:59.323 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:06:59.324 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:06:59.324 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:06:59.324 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:06:59.324 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:06:59.324 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:06:59.791 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-03-29T04:06:59.793 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-03-29T04:06:59.793 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-29T04:06:59.885 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-29T04:06:59.917 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-29T04:06:59.984 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:06:59.984 INFO:tasks.ceph:role: osd.6 2024-03-29T04:06:59.984 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T04:06:59.984 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-29T04:07:00.059 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:07:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:07:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:07:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:07:00.059 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:07:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:07:00.059 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:07:00.059 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:07:00.060 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:07:00.060 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:07:00.545 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-03-29T04:07:00.547 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-03-29T04:07:00.547 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-29T04:07:00.664 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-29T04:07:00.696 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-29T04:07:00.763 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:07:00.763 INFO:tasks.ceph:role: osd.7 2024-03-29T04:07:00.764 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T04:07:00.764 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-29T04:07:00.838 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:07:00.838 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:07:00.838 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:07:00.838 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:07:00.838 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:07:00.839 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:07:00.839 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:07:00.839 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:07:00.839 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:07:00.839 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:07:01.359 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-03-29T04:07:01.362 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-03-29T04:07:01.362 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-29T04:07:01.496 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-29T04:07:01.529 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:07:01.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:01.652+0000 7f0f43c1e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-29T04:07:01.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:01.652+0000 7f0f43c1e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-29T04:07:01.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:01.652+0000 7f0f43c1e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T04:07:01.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:01.652+0000 7f0f43c1e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T04:07:01.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:01.652+0000 7f0f43c1e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T04:07:01.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:01.653+0000 7f0f43c1e640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-29T04:07:04.481 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-29T04:07:04.511 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:07:04.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:04.621+0000 7fd84f61b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-29T04:07:04.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:04.621+0000 7fd84f61b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-29T04:07:04.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:04.621+0000 7fd84f61b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T04:07:04.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:04.621+0000 7fd84f61b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T04:07:04.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:04.621+0000 7fd84f61b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T04:07:04.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:04.621+0000 7fd84f61b640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-29T04:07:07.453 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-29T04:07:07.483 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:07:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:07.592+0000 7fbb20817640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-29T04:07:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:07.592+0000 7fbb20817640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-29T04:07:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:07.592+0000 7fbb20817640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T04:07:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:07.592+0000 7fbb20817640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T04:07:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:07.592+0000 7fbb20817640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T04:07:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:07.592+0000 7fbb20817640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-29T04:07:10.066 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:07:10.069 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:07:10.418 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-29T04:07:10.447 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:07:10.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:10.557+0000 7fd6b0a8b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-29T04:07:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:10.558+0000 7fd6b0a8b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-29T04:07:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:10.558+0000 7fd6b0a8b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T04:07:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:10.558+0000 7fd6b0a8b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T04:07:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:10.558+0000 7fd6b0a8b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T04:07:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:10.558+0000 7fd6b0a8b640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-29T04:07:13.390 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-29T04:07:13.420 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-29T04:07:13.421 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:13.421 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-29T04:07:13.453 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:13.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-29T04:07:13.523 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:13.523 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-29T04:07:13.591 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:13.592 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-29T04:07:13.660 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:13.660 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-29T04:07:13.728 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:13.728 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-29T04:07:13.795 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:13.796 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-29T04:07:13.862 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:13.862 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-29T04:07:13.895 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:13.895 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-29T04:07:13.966 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:13.966 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-29T04:07:14.038 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:14.038 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-29T04:07:14.109 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:14.109 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-29T04:07:14.180 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:14.180 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-29T04:07:14.249 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:14.249 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-29T04:07:14.319 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:14.319 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-29T04:07:14.334 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:14.334 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-29T04:07:14.389 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:14.389 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-29T04:07:14.405 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:14.405 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-29T04:07:14.461 INFO:tasks.ceph:Adding keys to all mons... 2024-03-29T04:07:14.462 DEBUG:teuthology.orchestra.run.smithi097:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-29T04:07:14.464 DEBUG:teuthology.orchestra.run.smithi154:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-29T04:07:14.504 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.y] 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBSPgZmitfHIBAARJO706tkS1jlAHRtaoWd3Q== 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBSPgZmEj/dKRAAytsfrGa+X1yjseBjTlCkJw== 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBSPgZm2aQ0MRAAHHqNIpIdDOQscqREfWR5Qw== 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.0] 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBWPgZmMa1kIxAAtEFfOrYEUQYoHTAlyX1guw== 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.1] 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBZPgZmNrSeIhAAPAPvi9LSGTMEMTAFcsCqOg== 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.2] 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBcPgZmp3D+IBAAd/jYMEC4jy0Kq7cPERtTvg== 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.3] 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBfPgZmC7ngHxAAtLZmZoRdOw5GzLtgr5qjwQ== 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.x] 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBSPgZmEpVjJhAA73QsPidRWekJsp1yxqYzOQ== 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBSPgZmL0g2NhAA28cGRN0Fd4nFlOWU7PNtXg== 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBTPgZmaoAFAhAAeDOfrEhIg+cSNEiguyksXA== 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.4] 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBlPgZm6arzJhAAV8CWcM4jEJnd4AYUYrwIEw== 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.5] 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBoPgZmtv0SJRAAethE1tyjHhFfzzJ4yn4MNg== 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.6] 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBrPgZmkf9WIxAA1eCaLkdytaQEIc0DT60e+w== 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.7] 2024-03-29T04:07:14.506 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBuPgZmZnRLIRAAUMAP+Ux2za2z+Kuvx9EPPg== 2024-03-29T04:07:14.507 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-03-29T04:07:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBTPgZm8VqYBhAAmwz9Zhd1EMjy52y5RAJy3w== 2024-03-29T04:07:14.507 INFO:teuthology.orchestra.run.smithi097.stdout:[client.2] 2024-03-29T04:07:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBTPgZmLglsDBAAFvRmWjIVDH4e7vYpj54bNQ== 2024-03-29T04:07:14.507 INFO:teuthology.orchestra.run.smithi097.stdout:[client.1] 2024-03-29T04:07:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBTPgZmzTfpDxAAVWrNCx4mKvts114/QRML4A== 2024-03-29T04:07:14.507 INFO:teuthology.orchestra.run.smithi097.stdout:[client.3] 2024-03-29T04:07:14.507 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBTPgZm1zPTFRAAMh+bKVY1N263z+C/axveaQ== 2024-03-29T04:07:14.528 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.y] 2024-03-29T04:07:14.528 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBSPgZmitfHIBAARJO706tkS1jlAHRtaoWd3Q== 2024-03-29T04:07:14.528 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2024-03-29T04:07:14.528 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBSPgZmEj/dKRAAytsfrGa+X1yjseBjTlCkJw== 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBSPgZm2aQ0MRAAHHqNIpIdDOQscqREfWR5Qw== 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.0] 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBWPgZmMa1kIxAAtEFfOrYEUQYoHTAlyX1guw== 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.1] 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBZPgZmNrSeIhAAPAPvi9LSGTMEMTAFcsCqOg== 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.2] 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBcPgZmp3D+IBAAd/jYMEC4jy0Kq7cPERtTvg== 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.3] 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBfPgZmC7ngHxAAtLZmZoRdOw5GzLtgr5qjwQ== 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.x] 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBSPgZmEpVjJhAA73QsPidRWekJsp1yxqYzOQ== 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBSPgZmL0g2NhAA28cGRN0Fd4nFlOWU7PNtXg== 2024-03-29T04:07:14.529 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBTPgZmaoAFAhAAeDOfrEhIg+cSNEiguyksXA== 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.4] 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBlPgZm6arzJhAAV8CWcM4jEJnd4AYUYrwIEw== 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.5] 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBoPgZmtv0SJRAAethE1tyjHhFfzzJ4yn4MNg== 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.6] 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBrPgZmkf9WIxAA1eCaLkdytaQEIc0DT60e+w== 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.7] 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBuPgZmZnRLIRAAUMAP+Ux2za2z+Kuvx9EPPg== 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBTPgZm8VqYBhAAmwz9Zhd1EMjy52y5RAJy3w== 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout:[client.2] 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBTPgZmLglsDBAAFvRmWjIVDH4e7vYpj54bNQ== 2024-03-29T04:07:14.530 INFO:teuthology.orchestra.run.smithi154.stdout:[client.1] 2024-03-29T04:07:14.531 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBTPgZmzTfpDxAAVWrNCx4mKvts114/QRML4A== 2024-03-29T04:07:14.531 INFO:teuthology.orchestra.run.smithi154.stdout:[client.3] 2024-03-29T04:07:14.531 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBTPgZm1zPTFRAAMh+bKVY1N263z+C/axveaQ== 2024-03-29T04:07:14.532 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:07:14.551 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:07:14.625 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:07:14.627 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:07:14.681 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:07:14.683 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:07:14.739 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:14.741 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:14.793 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:14.795 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:14.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:14.849 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:14.901 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:14.903 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:14.956 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:07:14.958 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:07:15.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:07:15.012 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:07:15.066 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:07:15.068 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:07:15.123 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:15.125 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:15.177 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:15.179 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:15.231 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:15.233 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:15.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:15.288 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:07:15.340 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:07:15.343 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:07:15.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:07:15.402 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:07:15.457 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:07:15.460 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:07:15.515 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:07:15.517 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:07:15.574 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-29T04:07:15.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-29T04:07:15.601 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T04:07:15.698 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-29T04:07:15.725 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-29T04:07:15.752 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T04:07:15.850 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-29T04:07:15.877 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-29T04:07:15.943 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T04:07:16.042 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-29T04:07:16.069 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:07:16.071 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:07:16.123 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-29T04:07:16.123 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-29T04:07:16.123 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-29T04:07:16.125 INFO:tasks.ceph.mon.a:Started 2024-03-29T04:07:16.126 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-29T04:07:16.126 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-29T04:07:16.166 INFO:tasks.ceph.mon.b:Started 2024-03-29T04:07:16.166 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-29T04:07:16.166 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-29T04:07:16.168 INFO:tasks.ceph.mon.c:Started 2024-03-29T04:07:16.168 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-29T04:07:16.168 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-29T04:07:16.169 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-29T04:07:16.171 INFO:tasks.ceph.mgr.y:Started 2024-03-29T04:07:16.171 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-29T04:07:16.171 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-29T04:07:16.173 INFO:tasks.ceph.mgr.x:Started 2024-03-29T04:07:16.173 INFO:tasks.ceph:Setting crush tunables to default 2024-03-29T04:07:16.173 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd crush tunables default 2024-03-29T04:07:16.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.292+0000 7f22f961b640 1 Processor -- start 2024-03-29T04:07:16.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.292+0000 7f22f961b640 1 -- start start 2024-03-29T04:07:16.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.293+0000 7f22f961b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f22f40d0f70 0x7f22f40d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:16.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.293+0000 7f22f961b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f22f40d21e0 0x7f22f40d2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:16.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.293+0000 7f22f961b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f40d1890 con 0x7f22f40cf1b0 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.293+0000 7f22f961b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f22f40cf690 con 0x7f22f40d0f70 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.293+0000 7f22f961b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f22f40cf970 con 0x7f22f40d21e0 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.293+0000 7f22f2ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f22f40d0f70 0x7f22f40d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.293+0000 7f22f2ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f22f40d0f70 0x7f22f40d1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49634/0 (socket says 172.21.15.97:49634) 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.293+0000 7f22f2ffd640 1 -- 172.21.15.97:0/1747467124 learned_addr learned my addr 172.21.15.97:0/1747467124 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.293+0000 7f22f37fe640 1 --2- 172.21.15.97:0/1747467124 >> v2:172.21.15.154:3300/0 conn(0x7f22f40d21e0 0x7f22f40d2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.293+0000 7f22f2ffd640 1 -- 172.21.15.97:0/1747467124 >> v2:172.21.15.97:3300/0 conn(0x7f22f40d0f70 msgr2=0x7f22f40d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.293+0000 7f22f2ffd640 1 -- 172.21.15.97:0/1747467124 >> v2:172.21.15.97:3300/0 conn(0x7f22f40d0f70 msgr2=0x7f22f40d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.293+0000 7f22f2ffd640 1 --2- 172.21.15.97:0/1747467124 >> v2:172.21.15.97:3300/0 conn(0x7f22f40d0f70 0x7f22f40d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.294+0000 7f22f2ffd640 1 --2- 172.21.15.97:0/1747467124 >> v2:172.21.15.97:3300/0 conn(0x7f22f40d0f70 0x7f22f40d1350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.294+0000 7f22f37fe640 1 -- 172.21.15.97:0/1747467124 >> v2:172.21.15.154:3300/0 conn(0x7f22f40d21e0 msgr2=0x7f22f40d2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.294+0000 7f22f37fe640 1 -- 172.21.15.97:0/1747467124 >> v2:172.21.15.154:3300/0 conn(0x7f22f40d21e0 msgr2=0x7f22f40d2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.294+0000 7f22f37fe640 1 --2- 172.21.15.97:0/1747467124 >> v2:172.21.15.154:3300/0 conn(0x7f22f40d21e0 0x7f22f40d2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:07:16.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.294+0000 7f22f37fe640 1 --2- 172.21.15.97:0/1747467124 >> v2:172.21.15.154:3300/0 conn(0x7f22f40d21e0 0x7f22f40d2650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:07:16.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.457+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 581969483 0 0) 0x7f22f40d1890 con 0x7f22f40cf1b0 2024-03-29T04:07:16.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.457+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22dc003520 con 0x7f22f40cf1b0 2024-03-29T04:07:16.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.457+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2299565317 0 0) 0x7f22dc003520 con 0x7f22f40cf1b0 2024-03-29T04:07:16.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.457+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 >> v2:172.21.15.154:3300/0 conn(0x7f22f40d21e0 msgr2=0x7f22f40d2650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:07:16.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.457+0000 7f22f1ffb640 1 --2- 172.21.15.97:0/1747467124 >> v2:172.21.15.154:3300/0 conn(0x7f22f40d21e0 0x7f22f40d2650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:16.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.457+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 >> v2:172.21.15.97:3300/0 conn(0x7f22f40d0f70 msgr2=0x7f22f40d1350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:07:16.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.457+0000 7f22f1ffb640 1 --2- 172.21.15.97:0/1747467124 >> v2:172.21.15.97:3300/0 conn(0x7f22f40d0f70 0x7f22f40d1350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:16.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.457+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f40cfc50 con 0x7f22f40cf1b0 2024-03-29T04:07:16.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.457+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f22e80016e0 con 0x7f22f40cf1b0 2024-03-29T04:07:16.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 >> v1:172.21.15.97:6789/0 conn(0x7f22f40cf1b0 legacy=0x7f22f40d4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:16.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f1ffb640 1 --2- 172.21.15.97:0/1747467124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22dc0039b0 0x7f22dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:16.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f1ffb640 1 --2- 172.21.15.97:0/1747467124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22dc004800 0x7f22dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:16.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f1ffb640 1 --2- 172.21.15.97:0/1747467124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22dc0051c0 0x7f22dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:16.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f22f40cf690 con 0x7f22dc004800 2024-03-29T04:07:16.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f22f40cf970 con 0x7f22dc0051c0 2024-03-29T04:07:16.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc00c0c0 con 0x7f22dc0039b0 2024-03-29T04:07:16.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f27fc640 1 --2- 172.21.15.97:0/1747467124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22dc004800 0x7f22dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:16.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f37fe640 1 --2- 172.21.15.97:0/1747467124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22dc0051c0 0x7f22dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:16.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f2ffd640 1 --2- 172.21.15.97:0/1747467124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22dc0039b0 0x7f22dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:16.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f27fc640 1 -- 172.21.15.97:0/1747467124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22dc0039b0 msgr2=0x7f22dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:16.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f27fc640 1 --2- 172.21.15.97:0/1747467124 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22dc0039b0 0x7f22dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:16.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f27fc640 1 -- 172.21.15.97:0/1747467124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22dc0051c0 msgr2=0x7f22dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:16.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f27fc640 1 --2- 172.21.15.97:0/1747467124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22dc0051c0 0x7f22dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:16.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f27fc640 1 -- 172.21.15.97:0/1747467124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc003520 con 0x7f22dc004800 2024-03-29T04:07:16.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f37fe640 1 --2- 172.21.15.97:0/1747467124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22dc0051c0 0x7f22dc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:16.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.458+0000 7f22f37fe640 1 -- 172.21.15.97:0/1747467124 reap_dead start 2024-03-29T04:07:16.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.459+0000 7f22f27fc640 1 --2- 172.21.15.97:0/1747467124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22dc004800 0x7f22dc004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f22e800c190 tx=0x7f22dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba0e28d70719c0c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:16.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.459+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8007c60 con 0x7f22dc004800 2024-03-29T04:07:16.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.459+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e8030070 con 0x7f22dc004800 2024-03-29T04:07:16.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.459+0000 7f22f1ffb640 1 -- 172.21.15.97:0/1747467124 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e802c850 con 0x7f22dc004800 2024-03-29T04:07:16.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.459+0000 7f22f961b640 1 -- 172.21.15.97:0/1747467124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22dc004800 msgr2=0x7f22dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:16.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.459+0000 7f22f961b640 1 --2- 172.21.15.97:0/1747467124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22dc004800 0x7f22dc004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f22e800c190 tx=0x7f22dc005830 comp rx=0 tx=0).stop 2024-03-29T04:07:16.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.460+0000 7f22f961b640 1 -- 172.21.15.97:0/1747467124 shutdown_connections 2024-03-29T04:07:16.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.460+0000 7f22f961b640 1 --2- 172.21.15.97:0/1747467124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22dc004800 0x7f22dc004c80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.460+0000 7f22f961b640 1 -- 172.21.15.97:0/1747467124 >> 172.21.15.97:0/1747467124 conn(0x7f22f408bfc0 msgr2=0x7f22f40d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.460+0000 7f22f961b640 1 -- 172.21.15.97:0/1747467124 shutdown_connections 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.460+0000 7f22f961b640 1 -- 172.21.15.97:0/1747467124 wait complete. 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.461+0000 7f22f961b640 1 Processor -- start 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.461+0000 7f22f961b640 1 -- start start 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f961b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22dc004800 0x7f22f40b9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f961b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22f40b8f90 0x7f22f41e5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f961b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ca330 0x7f22f40ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f961b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f22f415fcd0 con 0x7f22f40ca330 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f961b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f22f40d0210 con 0x7f22dc004800 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f961b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f22f415eac0 con 0x7f22f40b8f90 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ca330 0x7f22f40ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ca330 0x7f22f40ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49652/0 (socket says 172.21.15.97:49652) 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f37fe640 1 -- 172.21.15.97:0/1549658879 learned_addr learned my addr 172.21.15.97:0/1549658879 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:16.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f27fc640 1 --2- 172.21.15.97:0/1549658879 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22f40b8f90 0x7f22f41e5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f2ffd640 1 --2- 172.21.15.97:0/1549658879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22dc004800 0x7f22f40b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f37fe640 1 -- 172.21.15.97:0/1549658879 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22f40b8f90 msgr2=0x7f22f41e5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f37fe640 1 --2- 172.21.15.97:0/1549658879 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22f40b8f90 0x7f22f41e5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f37fe640 1 -- 172.21.15.97:0/1549658879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22dc004800 msgr2=0x7f22f40b9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f37fe640 1 --2- 172.21.15.97:0/1549658879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22dc004800 0x7f22f40b9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f37fe640 1 -- 172.21.15.97:0/1549658879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f40cad20 con 0x7f22f40ca330 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f37fe640 1 --2- 172.21.15.97:0/1549658879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ca330 0x7f22f40ca7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f22f40c7ed0 tx=0x7f22ec004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22db7fe640 1 -- 172.21.15.97:0/1549658879 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec018020 con 0x7f22f40ca330 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22db7fe640 1 -- 172.21.15.97:0/1549658879 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ec00b500 con 0x7f22f40ca330 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f961b640 1 -- 172.21.15.97:0/1549658879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f415db80 con 0x7f22f40ca330 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22db7fe640 1 -- 172.21.15.97:0/1549658879 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec007aa0 con 0x7f22f40ca330 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22f961b640 1 -- 172.21.15.97:0/1549658879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f40b8790 con 0x7f22f40ca330 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22db7fe640 1 -- 172.21.15.97:0/1549658879 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f22ec0096c0 con 0x7f22f40ca330 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.462+0000 7f22db7fe640 1 -- 172.21.15.97:0/1549658879 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f22ec01d9e0 con 0x7f22f40ca330 2024-03-29T04:07:16.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.463+0000 7f22f961b640 1 -- 172.21.15.97:0/1549658879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f40a2500 con 0x7f22f40ca330 2024-03-29T04:07:16.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.467+0000 7f22db7fe640 1 -- 172.21.15.97:0/1549658879 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f22ec022020 con 0x7f22f40ca330 2024-03-29T04:07:16.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.531+0000 7f22f961b640 1 -- 172.21.15.97:0/1549658879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f22f40d0b40 con 0x7f22f40ca330 2024-03-29T04:07:16.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.532+0000 7f22db7fe640 1 -- 172.21.15.97:0/1549658879 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f22ec01d400 con 0x7f22f40ca330 2024-03-29T04:07:16.512 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2024-03-29T04:07:16.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.533+0000 7f22f961b640 1 -- 172.21.15.97:0/1549658879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ca330 msgr2=0x7f22f40ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:16.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.533+0000 7f22f961b640 1 --2- 172.21.15.97:0/1549658879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ca330 0x7f22f40ca7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f22f40c7ed0 tx=0x7f22ec004700 comp rx=0 tx=0).stop 2024-03-29T04:07:16.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.533+0000 7f22f961b640 1 -- 172.21.15.97:0/1549658879 shutdown_connections 2024-03-29T04:07:16.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.533+0000 7f22f961b640 1 --2- 172.21.15.97:0/1549658879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22f40ca330 0x7f22f40ca7e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:16.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.533+0000 7f22f961b640 1 --2- 172.21.15.97:0/1549658879 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22f40b8f90 0x7f22f41e5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:16.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.533+0000 7f22f961b640 1 --2- 172.21.15.97:0/1549658879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22dc004800 0x7f22f40b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:16.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.533+0000 7f22f961b640 1 -- 172.21.15.97:0/1549658879 >> 172.21.15.97:0/1549658879 conn(0x7f22f408bfc0 msgr2=0x7f22f406cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:16.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.533+0000 7f22f961b640 1 -- 172.21.15.97:0/1549658879 shutdown_connections 2024-03-29T04:07:16.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:16.533+0000 7f22f961b640 1 -- 172.21.15.97:0/1549658879 wait complete. 2024-03-29T04:07:16.523 INFO:tasks.ceph:check_enable_crimson: False 2024-03-29T04:07:16.523 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-29T04:07:16.524 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:16.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-29T04:07:16.550 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:16.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-29T04:07:16.617 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:16.617 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-29T04:07:16.684 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:16.684 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-29T04:07:16.751 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:16.752 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-29T04:07:16.779 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:16.780 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-29T04:07:16.847 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:16.847 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-29T04:07:16.913 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:16.913 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-29T04:07:16.915 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:16.935+0000 7f659be1a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-29T04:07:16.929 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:16.950+0000 7f083402b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-29T04:07:16.942 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new c6327747-649d-4d0b-9cbe-78f197b0a784 0 2024-03-29T04:07:16.972 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:16.992+0000 7f659be1a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-29T04:07:16.983 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:17.003+0000 7f083402b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-29T04:07:17.073 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.093+0000 7fa07359e640 1 Processor -- start 2024-03-29T04:07:17.073 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.093+0000 7fa07359e640 1 -- start start 2024-03-29T04:07:17.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.094+0000 7fa07359e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa07409aa30 0x7fa0740b0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.094+0000 7fa07359e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0740af550 con 0x7fa0740ae880 2024-03-29T04:07:17.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.094+0000 7fa07359e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0740af830 con 0x7fa074072c60 2024-03-29T04:07:17.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.094+0000 7fa07359e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fa0740afe80 con 0x7fa07409aa30 2024-03-29T04:07:17.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.094+0000 7fa071d9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa07409aa30 0x7fa0740b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.094+0000 7fa071d9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa07409aa30 0x7fa0740b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36774/0 (socket says 172.21.15.154:36774) 2024-03-29T04:07:17.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.094+0000 7fa071d9b640 1 -- 172.21.15.154:0/2783609905 learned_addr learned my addr 172.21.15.154:0/2783609905 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:17.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.095+0000 7fa071d9b640 1 -- 172.21.15.154:0/2783609905 >> v1:172.21.15.154:6789/0 conn(0x7fa074072c60 legacy=0x7fa0740b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.095+0000 7fa071d9b640 1 -- 172.21.15.154:0/2783609905 >> v1:172.21.15.97:6789/0 conn(0x7fa0740ae880 legacy=0x7fa0740aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.095+0000 7fa071d9b640 1 -- 172.21.15.154:0/2783609905 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0740afb10 con 0x7fa07409aa30 2024-03-29T04:07:17.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.095+0000 7fa071d9b640 1 --2- 172.21.15.154:0/2783609905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa07409aa30 0x7fa0740b0310 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009870 tx=0x7fa05c02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=651131fa038d102e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:17.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.095+0000 7fa07159a640 1 -- 172.21.15.154:0/2783609905 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c034070 con 0x7fa07409aa30 2024-03-29T04:07:17.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.096+0000 7fa07159a640 1 -- 172.21.15.154:0/2783609905 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05c030860 con 0x7fa07409aa30 2024-03-29T04:07:17.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.096+0000 7fa07159a640 1 -- 172.21.15.154:0/2783609905 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c030c30 con 0x7fa07409aa30 2024-03-29T04:07:17.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.096+0000 7fa07359e640 1 -- 172.21.15.154:0/2783609905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa07409aa30 msgr2=0x7fa0740b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.096+0000 7fa07359e640 1 --2- 172.21.15.154:0/2783609905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa07409aa30 0x7fa0740b0310 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009870 tx=0x7fa05c02b9a0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.096+0000 7fa07359e640 1 -- 172.21.15.154:0/2783609905 shutdown_connections 2024-03-29T04:07:17.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.096+0000 7fa07359e640 1 --2- 172.21.15.154:0/2783609905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa07409aa30 0x7fa0740b0310 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.096+0000 7fa07359e640 1 -- 172.21.15.154:0/2783609905 >> 172.21.15.154:0/2783609905 conn(0x7fa0740b6800 msgr2=0x7fa0740b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:17.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.096+0000 7fa07359e640 1 -- 172.21.15.154:0/2783609905 shutdown_connections 2024-03-29T04:07:17.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.096+0000 7fa07359e640 1 -- 172.21.15.154:0/2783609905 wait complete. 2024-03-29T04:07:17.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.097+0000 7fa07359e640 1 Processor -- start 2024-03-29T04:07:17.078 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.098+0000 7fa07359e640 1 -- start start 2024-03-29T04:07:17.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.098+0000 7fa07359e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa074072c60 0x7fa074151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.098+0000 7fa07359e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa07409aa30 0x7fa07406b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.098+0000 7fa07359e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0740ae880 0x7fa07406ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.098+0000 7fa07359e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa07415b650 con 0x7fa074072c60 2024-03-29T04:07:17.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.098+0000 7fa07359e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa0740b1340 con 0x7fa07409aa30 2024-03-29T04:07:17.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.098+0000 7fa07359e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa07415cb20 con 0x7fa0740ae880 2024-03-29T04:07:17.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.098+0000 7fa071d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa07409aa30 0x7fa07406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.098+0000 7fa071d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa07409aa30 0x7fa07406b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36776/0 (socket says 172.21.15.154:36776) 2024-03-29T04:07:17.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.098+0000 7fa071d9b640 1 -- 172.21.15.154:0/1859709817 learned_addr learned my addr 172.21.15.154:0/1859709817 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:17.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.098+0000 7fa072d9d640 1 --2- 172.21.15.154:0/1859709817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0740ae880 0x7fa07406ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.098+0000 7fa07259c640 1 --2- 172.21.15.154:0/1859709817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa074072c60 0x7fa074151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa071d9b640 1 -- 172.21.15.154:0/1859709817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0740ae880 msgr2=0x7fa07406ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa071d9b640 1 --2- 172.21.15.154:0/1859709817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0740ae880 0x7fa07406ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa071d9b640 1 -- 172.21.15.154:0/1859709817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa074072c60 msgr2=0x7fa074151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa071d9b640 1 --2- 172.21.15.154:0/1859709817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa074072c60 0x7fa074151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa071d9b640 1 -- 172.21.15.154:0/1859709817 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07415bf10 con 0x7fa07409aa30 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa072d9d640 1 --2- 172.21.15.154:0/1859709817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0740ae880 0x7fa07406ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa071d9b640 1 --2- 172.21.15.154:0/1859709817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa07409aa30 0x7fa07406b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa05c000c00 tx=0x7fa05c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa06affd640 1 -- 172.21.15.154:0/1859709817 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c048070 con 0x7fa07409aa30 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa06affd640 1 -- 172.21.15.154:0/1859709817 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05c039d90 con 0x7fa07409aa30 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa07359e640 1 -- 172.21.15.154:0/1859709817 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07415c1f0 con 0x7fa07409aa30 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa06affd640 1 -- 172.21.15.154:0/1859709817 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c034040 con 0x7fa07409aa30 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa07359e640 1 -- 172.21.15.154:0/1859709817 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07415c7b0 con 0x7fa07409aa30 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa06affd640 1 -- 172.21.15.154:0/1859709817 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa05c04b070 con 0x7fa07409aa30 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.099+0000 7fa06affd640 1 -- 172.21.15.154:0/1859709817 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fa05c04d440 con 0x7fa07409aa30 2024-03-29T04:07:17.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.100+0000 7fa07359e640 1 -- 172.21.15.154:0/1859709817 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa07404f8f0 con 0x7fa07409aa30 2024-03-29T04:07:17.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.102+0000 7fa06affd640 1 -- 172.21.15.154:0/1859709817 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa05c030720 con 0x7fa07409aa30 2024-03-29T04:07:17.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.174+0000 7fa07359e640 1 -- 172.21.15.154:0/1859709817 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c6327747-649d-4d0b-9cbe-78f197b0a784", "id": 0} v 0) -- 0x7fa0740d0ec0 con 0x7fa07409aa30 2024-03-29T04:07:17.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.177+0000 7fa06affd640 1 -- 172.21.15.154:0/1859709817 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c6327747-649d-4d0b-9cbe-78f197b0a784", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fa05c043630 con 0x7fa07409aa30 2024-03-29T04:07:17.157 INFO:teuthology.orchestra.run.smithi154.stdout:0 2024-03-29T04:07:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.178+0000 7fa07359e640 1 -- 172.21.15.154:0/1859709817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa07409aa30 msgr2=0x7fa07406b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.178+0000 7fa07359e640 1 --2- 172.21.15.154:0/1859709817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa07409aa30 0x7fa07406b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa05c000c00 tx=0x7fa05c004730 comp rx=0 tx=0).stop 2024-03-29T04:07:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.178+0000 7fa07359e640 1 -- 172.21.15.154:0/1859709817 shutdown_connections 2024-03-29T04:07:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.178+0000 7fa07359e640 1 --2- 172.21.15.154:0/1859709817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0740ae880 0x7fa07406ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.178+0000 7fa07359e640 1 --2- 172.21.15.154:0/1859709817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa07409aa30 0x7fa07406b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.178+0000 7fa07359e640 1 --2- 172.21.15.154:0/1859709817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa074072c60 0x7fa074151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.178+0000 7fa07359e640 1 -- 172.21.15.154:0/1859709817 >> 172.21.15.154:0/1859709817 conn(0x7fa0740b6800 msgr2=0x7fa0740b5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.178+0000 7fa07359e640 1 -- 172.21.15.154:0/1859709817 shutdown_connections 2024-03-29T04:07:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.178+0000 7fa07359e640 1 -- 172.21.15.154:0/1859709817 wait complete. 2024-03-29T04:07:17.169 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 176691c0-02fd-40e8-abb2-115fce34d4ab 1 2024-03-29T04:07:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.291+0000 7fbf79688640 1 Processor -- start 2024-03-29T04:07:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.291+0000 7fbf79688640 1 -- start start 2024-03-29T04:07:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.291+0000 7fbf79688640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf7415e920 0x7fbf740d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.292+0000 7fbf79688640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf7415ed00 con 0x7fbf740d16a0 2024-03-29T04:07:17.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.292+0000 7fbf79688640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf740d21e0 con 0x7fbf740cfde0 2024-03-29T04:07:17.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.292+0000 7fbf79688640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbf740d37c0 con 0x7fbf7415e920 2024-03-29T04:07:17.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.292+0000 7fbf727fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbf740cfde0 0x7fbf740d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46986/0 (socket says 172.21.15.154:46986) 2024-03-29T04:07:17.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.292+0000 7fbf727fc640 1 -- 172.21.15.154:0/4084621227 learned_addr learned my addr 172.21.15.154:0/4084621227 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-29T04:07:17.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.292+0000 7fbf72ffd640 1 --2- 172.21.15.154:0/4084621227 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf7415e920 0x7fbf740d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.292+0000 7fbf71ffb640 1 -- 172.21.15.154:0/4084621227 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1225828804 0 0) 0x7fbf740d21e0 con 0x7fbf740cfde0 2024-03-29T04:07:17.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.292+0000 7fbf71ffb640 1 -- 172.21.15.154:0/4084621227 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf5c003520 con 0x7fbf740cfde0 2024-03-29T04:07:17.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.292+0000 7fbf72ffd640 1 -- 172.21.15.154:0/4084621227 >> v1:172.21.15.154:6789/0 conn(0x7fbf740cfde0 legacy=0x7fbf740d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.292+0000 7fbf72ffd640 1 -- 172.21.15.154:0/4084621227 >> v1:172.21.15.97:6789/0 conn(0x7fbf740d16a0 legacy=0x7fbf7406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.292+0000 7fbf72ffd640 1 -- 172.21.15.154:0/4084621227 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf740d24c0 con 0x7fbf7415e920 2024-03-29T04:07:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.293+0000 7fbf72ffd640 1 --2- 172.21.15.154:0/4084621227 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf7415e920 0x7fbf740d3280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbf640097d0 tx=0x7fbf6402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c601968e0c838aa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.293+0000 7fbf71ffb640 1 -- 172.21.15.154:0/4084621227 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64035020 con 0x7fbf7415e920 2024-03-29T04:07:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.293+0000 7fbf71ffb640 1 -- 172.21.15.154:0/4084621227 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf64031890 con 0x7fbf7415e920 2024-03-29T04:07:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.293+0000 7fbf71ffb640 1 -- 172.21.15.154:0/4084621227 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64031c60 con 0x7fbf7415e920 2024-03-29T04:07:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.294+0000 7fbf79688640 1 -- 172.21.15.154:0/4084621227 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf7415e920 msgr2=0x7fbf740d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.294+0000 7fbf79688640 1 --2- 172.21.15.154:0/4084621227 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf7415e920 0x7fbf740d3280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbf640097d0 tx=0x7fbf6402b930 comp rx=0 tx=0).stop 2024-03-29T04:07:17.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.294+0000 7fbf79688640 1 -- 172.21.15.154:0/4084621227 shutdown_connections 2024-03-29T04:07:17.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.294+0000 7fbf79688640 1 --2- 172.21.15.154:0/4084621227 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf7415e920 0x7fbf740d3280 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.294+0000 7fbf79688640 1 -- 172.21.15.154:0/4084621227 >> 172.21.15.154:0/4084621227 conn(0x7fbf7408bfc0 msgr2=0x7fbf740d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:17.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.294+0000 7fbf79688640 1 -- 172.21.15.154:0/4084621227 shutdown_connections 2024-03-29T04:07:17.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.294+0000 7fbf79688640 1 -- 172.21.15.154:0/4084621227 wait complete. 2024-03-29T04:07:17.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.295+0000 7fbf79688640 1 Processor -- start 2024-03-29T04:07:17.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.295+0000 7fbf79688640 1 -- start start 2024-03-29T04:07:17.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf79688640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf740cfde0 0x7fbf7414e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf79688640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf740d16a0 0x7fbf740cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf79688640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf7415e920 0x7fbf740b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf79688640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf740d2d60 con 0x7fbf740d16a0 2024-03-29T04:07:17.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf79688640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbf740d2a80 con 0x7fbf740cfde0 2024-03-29T04:07:17.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf79688640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbf740d4f50 con 0x7fbf7415e920 2024-03-29T04:07:17.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf72ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf740cfde0 0x7fbf7414e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf72ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf740cfde0 0x7fbf7414e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36788/0 (socket says 172.21.15.154:36788) 2024-03-29T04:07:17.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf72ffd640 1 -- 172.21.15.154:0/4257965510 learned_addr learned my addr 172.21.15.154:0/4257965510 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf737fe640 1 --2- 172.21.15.154:0/4257965510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf7415e920 0x7fbf740b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf727fc640 1 --2- 172.21.15.154:0/4257965510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf740d16a0 0x7fbf740cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf72ffd640 1 -- 172.21.15.154:0/4257965510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf7415e920 msgr2=0x7fbf740b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf72ffd640 1 --2- 172.21.15.154:0/4257965510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf7415e920 0x7fbf740b6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf72ffd640 1 -- 172.21.15.154:0/4257965510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf740d16a0 msgr2=0x7fbf740cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf72ffd640 1 --2- 172.21.15.154:0/4257965510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf740d16a0 0x7fbf740cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf72ffd640 1 -- 172.21.15.154:0/4257965510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf740b85c0 con 0x7fbf740cfde0 2024-03-29T04:07:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf72ffd640 1 --2- 172.21.15.154:0/4257965510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf740cfde0 0x7fbf7414e460 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbf64034040 tx=0x7fbf640046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf5b7fe640 1 -- 172.21.15.154:0/4257965510 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64040070 con 0x7fbf740cfde0 2024-03-29T04:07:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf5b7fe640 1 -- 172.21.15.154:0/4257965510 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf64030d20 con 0x7fbf740cfde0 2024-03-29T04:07:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf5b7fe640 1 -- 172.21.15.154:0/4257965510 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64035040 con 0x7fbf740cfde0 2024-03-29T04:07:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf79688640 1 -- 172.21.15.154:0/4257965510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf740b88a0 con 0x7fbf740cfde0 2024-03-29T04:07:17.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.296+0000 7fbf79688640 1 -- 172.21.15.154:0/4257965510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf740b8e60 con 0x7fbf740cfde0 2024-03-29T04:07:17.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.297+0000 7fbf5b7fe640 1 -- 172.21.15.154:0/4257965510 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbf6404b060 con 0x7fbf740cfde0 2024-03-29T04:07:17.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.297+0000 7fbf5b7fe640 1 -- 172.21.15.154:0/4257965510 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fbf6404d9e0 con 0x7fbf740cfde0 2024-03-29T04:07:17.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.297+0000 7fbf79688640 1 -- 172.21.15.154:0/4257965510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf740a2560 con 0x7fbf740cfde0 2024-03-29T04:07:17.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.299+0000 7fbf5b7fe640 1 -- 172.21.15.154:0/4257965510 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbf64052020 con 0x7fbf740cfde0 2024-03-29T04:07:17.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.372+0000 7fbf79688640 1 -- 172.21.15.154:0/4257965510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "176691c0-02fd-40e8-abb2-115fce34d4ab", "id": 1} v 0) -- 0x7fbf7404f9d0 con 0x7fbf740cfde0 2024-03-29T04:07:17.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.375+0000 7fbf5b7fe640 1 -- 172.21.15.154:0/4257965510 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "176691c0-02fd-40e8-abb2-115fce34d4ab", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fbf6402d040 con 0x7fbf740cfde0 2024-03-29T04:07:17.355 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-03-29T04:07:17.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.376+0000 7fbf79688640 1 -- 172.21.15.154:0/4257965510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf740cfde0 msgr2=0x7fbf7414e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.376+0000 7fbf79688640 1 --2- 172.21.15.154:0/4257965510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf740cfde0 0x7fbf7414e460 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbf64034040 tx=0x7fbf640046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.376+0000 7fbf79688640 1 -- 172.21.15.154:0/4257965510 shutdown_connections 2024-03-29T04:07:17.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.376+0000 7fbf79688640 1 --2- 172.21.15.154:0/4257965510 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbf7415e920 0x7fbf740b6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.376+0000 7fbf79688640 1 --2- 172.21.15.154:0/4257965510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf740d16a0 0x7fbf740cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.376+0000 7fbf79688640 1 --2- 172.21.15.154:0/4257965510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf740cfde0 0x7fbf7414e460 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.376+0000 7fbf79688640 1 -- 172.21.15.154:0/4257965510 >> 172.21.15.154:0/4257965510 conn(0x7fbf7408bfc0 msgr2=0x7fbf740d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:17.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.376+0000 7fbf79688640 1 -- 172.21.15.154:0/4257965510 shutdown_connections 2024-03-29T04:07:17.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.376+0000 7fbf79688640 1 -- 172.21.15.154:0/4257965510 wait complete. 2024-03-29T04:07:17.367 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 306572ef-01fe-44d9-8488-61fc93883d50 2 2024-03-29T04:07:17.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.489+0000 7fdc8624e640 1 Processor -- start 2024-03-29T04:07:17.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.489+0000 7fdc8624e640 1 -- start start 2024-03-29T04:07:17.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc8624e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdc8015b6c0 0x7fdc800b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc8624e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8015baa0 0x7fdc800b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc8624e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc800b0b50 con 0x7fdc800b5b90 2024-03-29T04:07:17.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc8624e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fdc800aeb50 con 0x7fdc8015b6c0 2024-03-29T04:07:17.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc8624e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdc800b3c50 con 0x7fdc8015baa0 2024-03-29T04:07:17.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc7effd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8015baa0 0x7fdc800b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc7f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdc8015b6c0 0x7fdc800b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc7effd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8015baa0 0x7fdc800b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:57580/0 (socket says 172.21.15.154:57580) 2024-03-29T04:07:17.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc7f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdc8015b6c0 0x7fdc800b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36796/0 (socket says 172.21.15.154:36796) 2024-03-29T04:07:17.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc7effd640 1 -- 172.21.15.154:0/2704304756 learned_addr learned my addr 172.21.15.154:0/2704304756 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:17.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc7f7fe640 1 -- 172.21.15.154:0/2704304756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8015baa0 msgr2=0x7fdc800b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:17.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc7f7fe640 1 --2- 172.21.15.154:0/2704304756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8015baa0 0x7fdc800b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc7f7fe640 1 -- 172.21.15.154:0/2704304756 >> v1:172.21.15.154:6789/0 conn(0x7fdc800b5b90 legacy=0x7fdc8015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc7f7fe640 1 -- 172.21.15.154:0/2704304756 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc800aed80 con 0x7fdc8015b6c0 2024-03-29T04:07:17.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.490+0000 7fdc7effd640 1 --2- 172.21.15.154:0/2704304756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8015baa0 0x7fdc800b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:17.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.491+0000 7fdc7f7fe640 1 --2- 172.21.15.154:0/2704304756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc8015b6c0 0x7fdc800b4340 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009d90 tx=0x7fdc7002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e6a9f127a3925c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:17.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.491+0000 7fdc7e7fc640 1 -- 172.21.15.154:0/2704304756 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70039070 con 0x7fdc8015b6c0 2024-03-29T04:07:17.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.491+0000 7fdc7e7fc640 1 -- 172.21.15.154:0/2704304756 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc7002cc50 con 0x7fdc8015b6c0 2024-03-29T04:07:17.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.491+0000 7fdc7e7fc640 1 -- 172.21.15.154:0/2704304756 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70034440 con 0x7fdc8015b6c0 2024-03-29T04:07:17.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.492+0000 7fdc8624e640 1 -- 172.21.15.154:0/2704304756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc8015b6c0 msgr2=0x7fdc800b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.492+0000 7fdc8624e640 1 --2- 172.21.15.154:0/2704304756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc8015b6c0 0x7fdc800b4340 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009d90 tx=0x7fdc7002bea0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.492+0000 7fdc8624e640 1 -- 172.21.15.154:0/2704304756 shutdown_connections 2024-03-29T04:07:17.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.492+0000 7fdc8624e640 1 --2- 172.21.15.154:0/2704304756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8015baa0 0x7fdc800b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.492+0000 7fdc8624e640 1 --2- 172.21.15.154:0/2704304756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc8015b6c0 0x7fdc800b4340 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.492+0000 7fdc8624e640 1 -- 172.21.15.154:0/2704304756 >> 172.21.15.154:0/2704304756 conn(0x7fdc800b6a80 msgr2=0x7fdc800b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:17.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.492+0000 7fdc8624e640 1 -- 172.21.15.154:0/2704304756 shutdown_connections 2024-03-29T04:07:17.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.492+0000 7fdc8624e640 1 -- 172.21.15.154:0/2704304756 wait complete. 2024-03-29T04:07:17.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.493+0000 7fdc8624e640 1 Processor -- start 2024-03-29T04:07:17.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.493+0000 7fdc8624e640 1 -- start start 2024-03-29T04:07:17.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.494+0000 7fdc8624e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc800b5b90 0x7fdc8015e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.494+0000 7fdc8624e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8015b6c0 0x7fdc8014d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.494+0000 7fdc8624e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc8015baa0 0x7fdc8014fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.494+0000 7fdc8624e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdc800af620 con 0x7fdc8015baa0 2024-03-29T04:07:17.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.494+0000 7fdc8624e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdc800af340 con 0x7fdc800b5b90 2024-03-29T04:07:17.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.494+0000 7fdc8624e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdc8006e9d0 con 0x7fdc8015b6c0 2024-03-29T04:07:17.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.494+0000 7fdc7f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc800b5b90 0x7fdc8015e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.494+0000 7fdc7effd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8015b6c0 0x7fdc8014d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.494+0000 7fdc7f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc800b5b90 0x7fdc8015e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36806/0 (socket says 172.21.15.154:36806) 2024-03-29T04:07:17.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.494+0000 7fdc7f7fe640 1 -- 172.21.15.154:0/818740088 learned_addr learned my addr 172.21.15.154:0/818740088 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:17.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.494+0000 7fdc7ffff640 1 --2- 172.21.15.154:0/818740088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc8015baa0 0x7fdc8014fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc7f7fe640 1 -- 172.21.15.154:0/818740088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8015b6c0 msgr2=0x7fdc8014d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc7f7fe640 1 --2- 172.21.15.154:0/818740088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8015b6c0 0x7fdc8014d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc7f7fe640 1 -- 172.21.15.154:0/818740088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc8015baa0 msgr2=0x7fdc8014fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc7f7fe640 1 --2- 172.21.15.154:0/818740088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc8015baa0 0x7fdc8014fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc7f7fe640 1 -- 172.21.15.154:0/818740088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8015d220 con 0x7fdc800b5b90 2024-03-29T04:07:17.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc7ffff640 1 --2- 172.21.15.154:0/818740088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc8015baa0 0x7fdc8014fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:17.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc7f7fe640 1 --2- 172.21.15.154:0/818740088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc800b5b90 0x7fdc8015e2c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009d90 tx=0x7fdc70004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:17.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc67fff640 1 -- 172.21.15.154:0/818740088 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70044020 con 0x7fdc800b5b90 2024-03-29T04:07:17.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc67fff640 1 -- 172.21.15.154:0/818740088 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc7002ca50 con 0x7fdc800b5b90 2024-03-29T04:07:17.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc8624e640 1 -- 172.21.15.154:0/818740088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8015d500 con 0x7fdc800b5b90 2024-03-29T04:07:17.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc67fff640 1 -- 172.21.15.154:0/818740088 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70039040 con 0x7fdc800b5b90 2024-03-29T04:07:17.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc8624e640 1 -- 172.21.15.154:0/818740088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8015dac0 con 0x7fdc800b5b90 2024-03-29T04:07:17.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc67fff640 1 -- 172.21.15.154:0/818740088 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdc70005d10 con 0x7fdc800b5b90 2024-03-29T04:07:17.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.495+0000 7fdc67fff640 1 -- 172.21.15.154:0/818740088 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fdc70040c00 con 0x7fdc800b5b90 2024-03-29T04:07:17.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.496+0000 7fdc8624e640 1 -- 172.21.15.154:0/818740088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc8009f6a0 con 0x7fdc800b5b90 2024-03-29T04:07:17.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.500+0000 7fdc67fff640 1 -- 172.21.15.154:0/818740088 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdc7004e070 con 0x7fdc800b5b90 2024-03-29T04:07:17.531 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:17.550+0000 7f083402b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-29T04:07:17.535 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:17.556+0000 7f659be1a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-29T04:07:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.572+0000 7fdc8624e640 1 -- 172.21.15.154:0/818740088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "306572ef-01fe-44d9-8488-61fc93883d50", "id": 2} v 0) -- 0x7fdc8004f9d0 con 0x7fdc800b5b90 2024-03-29T04:07:17.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.578+0000 7fdc67fff640 1 -- 172.21.15.154:0/818740088 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "306572ef-01fe-44d9-8488-61fc93883d50", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fdc7002cc00 con 0x7fdc800b5b90 2024-03-29T04:07:17.558 INFO:teuthology.orchestra.run.smithi154.stdout:2 2024-03-29T04:07:17.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.579+0000 7fdc8624e640 1 -- 172.21.15.154:0/818740088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc800b5b90 msgr2=0x7fdc8015e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.579+0000 7fdc8624e640 1 --2- 172.21.15.154:0/818740088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc800b5b90 0x7fdc8015e2c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009d90 tx=0x7fdc70004a40 comp rx=0 tx=0).stop 2024-03-29T04:07:17.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.579+0000 7fdc8624e640 1 -- 172.21.15.154:0/818740088 shutdown_connections 2024-03-29T04:07:17.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.579+0000 7fdc8624e640 1 --2- 172.21.15.154:0/818740088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc8015baa0 0x7fdc8014fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.579+0000 7fdc8624e640 1 --2- 172.21.15.154:0/818740088 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc8015b6c0 0x7fdc8014d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.579+0000 7fdc8624e640 1 --2- 172.21.15.154:0/818740088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc800b5b90 0x7fdc8015e2c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.579+0000 7fdc8624e640 1 -- 172.21.15.154:0/818740088 >> 172.21.15.154:0/818740088 conn(0x7fdc800b6a80 msgr2=0x7fdc801563e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:17.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.579+0000 7fdc8624e640 1 -- 172.21.15.154:0/818740088 shutdown_connections 2024-03-29T04:07:17.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.579+0000 7fdc8624e640 1 -- 172.21.15.154:0/818740088 wait complete. 2024-03-29T04:07:17.570 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 59b4d556-3332-4c8e-b6f7-231ed9d64f7b 3 2024-03-29T04:07:17.584 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:17.605+0000 7f083402b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-29T04:07:17.594 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:17.614+0000 7f659be1a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-29T04:07:17.628 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:17.649+0000 7f083402b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-29T04:07:17.641 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:17.662+0000 7f659be1a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-29T04:07:17.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.691+0000 7f523257c640 1 Processor -- start 2024-03-29T04:07:17.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.691+0000 7f523257c640 1 -- start start 2024-03-29T04:07:17.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.692+0000 7f523257c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f522c0d03f0 0x7f522c0b0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.692+0000 7f523257c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f522c15eb90 0x7f522c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.692+0000 7f523257c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f522c0d0890 con 0x7f522c06b510 2024-03-29T04:07:17.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.692+0000 7f523257c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f522c0d0b70 con 0x7f522c0d03f0 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.692+0000 7f523257c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f522c0d0e50 con 0x7f522c15eb90 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.692+0000 7f522b7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f522c0d03f0 0x7f522c0b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.692+0000 7f5230b19640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f522c15eb90 0x7f522c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.692+0000 7f522b7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f522c0d03f0 0x7f522c0b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36816/0 (socket says 172.21.15.154:36816) 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.692+0000 7f5230b19640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f522c15eb90 0x7f522c0ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:57596/0 (socket says 172.21.15.154:57596) 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.692+0000 7f522b7fe640 1 -- 172.21.15.154:0/1810536654 learned_addr learned my addr 172.21.15.154:0/1810536654 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.693+0000 7f5230b19640 1 -- 172.21.15.154:0/1810536654 >> v2:172.21.15.154:3300/0 conn(0x7f522c0d03f0 msgr2=0x7f522c0b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.693+0000 7f5230b19640 1 --2- 172.21.15.154:0/1810536654 >> v2:172.21.15.154:3300/0 conn(0x7f522c0d03f0 0x7f522c0b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.693+0000 7f5230b19640 1 -- 172.21.15.154:0/1810536654 >> v1:172.21.15.154:6789/0 conn(0x7f522c06b510 legacy=0x7f522c06b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.693+0000 7f5230b19640 1 -- 172.21.15.154:0/1810536654 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522c0cde90 con 0x7f522c15eb90 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.693+0000 7f522b7fe640 1 --2- 172.21.15.154:0/1810536654 >> v2:172.21.15.154:3300/0 conn(0x7f522c0d03f0 0x7f522c0b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.693+0000 7f5230b19640 1 --2- 172.21.15.154:0/1810536654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f522c15eb90 0x7f522c0ce3b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5224009d90 tx=0x7f522402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93399f0f8b0f4b8a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.694+0000 7f522affd640 1 -- 172.21.15.154:0/1810536654 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224039070 con 0x7f522c15eb90 2024-03-29T04:07:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.694+0000 7f522affd640 1 -- 172.21.15.154:0/1810536654 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f522402cbc0 con 0x7f522c15eb90 2024-03-29T04:07:17.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.694+0000 7f522affd640 1 -- 172.21.15.154:0/1810536654 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224034440 con 0x7f522c15eb90 2024-03-29T04:07:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.694+0000 7f523257c640 1 -- 172.21.15.154:0/1810536654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f522c15eb90 msgr2=0x7f522c0ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.694+0000 7f523257c640 1 --2- 172.21.15.154:0/1810536654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f522c15eb90 0x7f522c0ce3b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5224009d90 tx=0x7f522402be10 comp rx=0 tx=0).stop 2024-03-29T04:07:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.694+0000 7f523257c640 1 -- 172.21.15.154:0/1810536654 shutdown_connections 2024-03-29T04:07:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.694+0000 7f523257c640 1 --2- 172.21.15.154:0/1810536654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f522c15eb90 0x7f522c0ce3b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.694+0000 7f523257c640 1 --2- 172.21.15.154:0/1810536654 >> v2:172.21.15.154:3300/0 conn(0x7f522c0d03f0 0x7f522c0b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.694+0000 7f523257c640 1 -- 172.21.15.154:0/1810536654 >> 172.21.15.154:0/1810536654 conn(0x7f522c0b2ed0 msgr2=0x7f522c0b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.694+0000 7f523257c640 1 -- 172.21.15.154:0/1810536654 shutdown_connections 2024-03-29T04:07:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.694+0000 7f523257c640 1 -- 172.21.15.154:0/1810536654 wait complete. 2024-03-29T04:07:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.695+0000 7f523257c640 1 Processor -- start 2024-03-29T04:07:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.696+0000 7f523257c640 1 -- start start 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.696+0000 7f523257c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f522c06b510 0x7f522c0c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.696+0000 7f523257c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f522c0d03f0 0x7f522c15b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.696+0000 7f523257c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f522c15eb90 0x7f522c157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.696+0000 7f523257c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f522c0cea70 con 0x7f522c15eb90 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.696+0000 7f523257c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f522c0ce8f0 con 0x7f522c0d03f0 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.696+0000 7f523257c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f522c0cb950 con 0x7f522c06b510 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f522b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f522c0d03f0 0x7f522c15b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f522b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f522c0d03f0 0x7f522c15b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36820/0 (socket says 172.21.15.154:36820) 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f522b7fe640 1 -- 172.21.15.154:0/2833643352 learned_addr learned my addr 172.21.15.154:0/2833643352 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f522bfff640 1 --2- 172.21.15.154:0/2833643352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f522c06b510 0x7f522c0c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f5230b19640 1 --2- 172.21.15.154:0/2833643352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f522c15eb90 0x7f522c157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f522b7fe640 1 -- 172.21.15.154:0/2833643352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f522c06b510 msgr2=0x7f522c0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f522b7fe640 1 --2- 172.21.15.154:0/2833643352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f522c06b510 0x7f522c0c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f522b7fe640 1 -- 172.21.15.154:0/2833643352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f522c15eb90 msgr2=0x7f522c157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f522b7fe640 1 --2- 172.21.15.154:0/2833643352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f522c15eb90 0x7f522c157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f522b7fe640 1 -- 172.21.15.154:0/2833643352 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522c0c7d60 con 0x7f522c0d03f0 2024-03-29T04:07:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f5230b19640 1 --2- 172.21.15.154:0/2833643352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f522c15eb90 0x7f522c157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f522b7fe640 1 --2- 172.21.15.154:0/2833643352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f522c0d03f0 0x7f522c15b100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f521c00e9d0 tx=0x7f521c00ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f5228ff9640 1 -- 172.21.15.154:0/2833643352 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c017070 con 0x7f522c0d03f0 2024-03-29T04:07:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f5228ff9640 1 -- 172.21.15.154:0/2833643352 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f521c00ca60 con 0x7f522c0d03f0 2024-03-29T04:07:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f5228ff9640 1 -- 172.21.15.154:0/2833643352 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c00ce10 con 0x7f522c0d03f0 2024-03-29T04:07:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f523257c640 1 -- 172.21.15.154:0/2833643352 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522c15c070 con 0x7f522c0d03f0 2024-03-29T04:07:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f523257c640 1 -- 172.21.15.154:0/2833643352 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522c15c630 con 0x7f522c0d03f0 2024-03-29T04:07:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f5228ff9640 1 -- 172.21.15.154:0/2833643352 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f521c01a070 con 0x7f522c0d03f0 2024-03-29T04:07:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.697+0000 7f5228ff9640 1 -- 172.21.15.154:0/2833643352 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f521c013470 con 0x7f522c0d03f0 2024-03-29T04:07:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.698+0000 7f523257c640 1 -- 172.21.15.154:0/2833643352 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f522c0b0e00 con 0x7f522c0d03f0 2024-03-29T04:07:17.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.700+0000 7f5228ff9640 1 -- 172.21.15.154:0/2833643352 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f521c006020 con 0x7f522c0d03f0 2024-03-29T04:07:17.682 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:17.702+0000 7f083402b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-29T04:07:17.699 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:17.719+0000 7f659be1a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-29T04:07:17.724 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:17.744+0000 7f083402b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-29T04:07:17.743 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:17.763+0000 7f659be1a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-29T04:07:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.773+0000 7f523257c640 1 -- 172.21.15.154:0/2833643352 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "59b4d556-3332-4c8e-b6f7-231ed9d64f7b", "id": 3} v 0) -- 0x7f522c04f9d0 con 0x7f522c0d03f0 2024-03-29T04:07:17.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.778+0000 7f5228ff9640 1 -- 172.21.15.154:0/2833643352 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "59b4d556-3332-4c8e-b6f7-231ed9d64f7b", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f521c006520 con 0x7f522c0d03f0 2024-03-29T04:07:17.758 INFO:teuthology.orchestra.run.smithi154.stdout:3 2024-03-29T04:07:17.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.779+0000 7f523257c640 1 -- 172.21.15.154:0/2833643352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f522c0d03f0 msgr2=0x7f522c15b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.779+0000 7f523257c640 1 --2- 172.21.15.154:0/2833643352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f522c0d03f0 0x7f522c15b100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f521c00e9d0 tx=0x7f521c00ee90 comp rx=0 tx=0).stop 2024-03-29T04:07:17.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.779+0000 7f523257c640 1 -- 172.21.15.154:0/2833643352 shutdown_connections 2024-03-29T04:07:17.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.779+0000 7f523257c640 1 --2- 172.21.15.154:0/2833643352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f522c15eb90 0x7f522c157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.779+0000 7f523257c640 1 --2- 172.21.15.154:0/2833643352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f522c0d03f0 0x7f522c15b100 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.779+0000 7f523257c640 1 --2- 172.21.15.154:0/2833643352 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f522c06b510 0x7f522c0c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.779+0000 7f523257c640 1 -- 172.21.15.154:0/2833643352 >> 172.21.15.154:0/2833643352 conn(0x7f522c0b2ed0 msgr2=0x7f522c0b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:17.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.779+0000 7f523257c640 1 -- 172.21.15.154:0/2833643352 shutdown_connections 2024-03-29T04:07:17.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.779+0000 7f523257c640 1 -- 172.21.15.154:0/2833643352 wait complete. 2024-03-29T04:07:17.769 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 1910d100-2ee1-4a4c-8eb5-427373f8d0c7 4 2024-03-29T04:07:17.840 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:17.860+0000 7f083402b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-29T04:07:17.866 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:17.887+0000 7f659be1a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-29T04:07:17.908 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:17.928+0000 7f083402b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-29T04:07:17.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.932+0000 7f39fb2c8640 1 Processor -- start 2024-03-29T04:07:17.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.932+0000 7f39fb2c8640 1 -- start start 2024-03-29T04:07:17.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39fb2c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39f413f860 0x7f39f406ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39fb2c8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f39f415d620 0x7f39f40add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39fb2c8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f406b960 con 0x7f39f40ae2b0 2024-03-29T04:07:17.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39fb2c8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f39f406bc40 con 0x7f39f415d620 2024-03-29T04:07:17.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39fb2c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f39f406bf20 con 0x7f39f413f860 2024-03-29T04:07:17.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39f9064640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39f413f860 0x7f39f406ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39f8863640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f39f415d620 0x7f39f40add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39f9064640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39f413f860 0x7f39f406ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:57626/0 (socket says 172.21.15.154:57626) 2024-03-29T04:07:17.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39f8863640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f39f415d620 0x7f39f40add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36824/0 (socket says 172.21.15.154:36824) 2024-03-29T04:07:17.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39f9064640 1 -- 172.21.15.154:0/3647035374 learned_addr learned my addr 172.21.15.154:0/3647035374 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:17.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39f9865640 1 --1- 172.21.15.154:0/3647035374 >> v1:172.21.15.154:6789/0 conn(0x7f39f40ae2b0 0x7f39f413fdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47016/0 (socket says 172.21.15.154:47016) 2024-03-29T04:07:17.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39f8863640 1 -- 172.21.15.154:0/3647035374 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39f413f860 msgr2=0x7f39f406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:17.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39f8863640 1 --2- 172.21.15.154:0/3647035374 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39f413f860 0x7f39f406ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39f8863640 1 -- 172.21.15.154:0/3647035374 >> v1:172.21.15.154:6789/0 conn(0x7f39f40ae2b0 legacy=0x7f39f413fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39f8863640 1 -- 172.21.15.154:0/3647035374 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f406c200 con 0x7f39f415d620 2024-03-29T04:07:17.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.933+0000 7f39f9064640 1 --2- 172.21.15.154:0/3647035374 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39f413f860 0x7f39f406ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:17.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.934+0000 7f39f8863640 1 --2- 172.21.15.154:0/3647035374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39f415d620 0x7f39f40add70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f39e40097a0 tx=0x7f39e402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=142f1721ee736c2f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:17.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.934+0000 7f39ebfff640 1 -- 172.21.15.154:0/3647035374 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e40315a0 con 0x7f39f415d620 2024-03-29T04:07:17.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.934+0000 7f39ebfff640 1 -- 172.21.15.154:0/3647035374 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e4037070 con 0x7f39f415d620 2024-03-29T04:07:17.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.934+0000 7f39ebfff640 1 -- 172.21.15.154:0/3647035374 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4031960 con 0x7f39f415d620 2024-03-29T04:07:17.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.935+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3647035374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39f415d620 msgr2=0x7f39f40add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.935+0000 7f39fb2c8640 1 --2- 172.21.15.154:0/3647035374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39f415d620 0x7f39f40add70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f39e40097a0 tx=0x7f39e402b840 comp rx=0 tx=0).stop 2024-03-29T04:07:17.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.935+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3647035374 shutdown_connections 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.935+0000 7f39fb2c8640 1 --2- 172.21.15.154:0/3647035374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39f415d620 0x7f39f40add70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.935+0000 7f39fb2c8640 1 --2- 172.21.15.154:0/3647035374 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39f413f860 0x7f39f406ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.935+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3647035374 >> 172.21.15.154:0/3647035374 conn(0x7f39f40d0bb0 msgr2=0x7f39f40d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.935+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3647035374 shutdown_connections 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.935+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3647035374 wait complete. 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.936+0000 7f39fb2c8640 1 Processor -- start 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.936+0000 7f39fb2c8640 1 -- start start 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39fb2c8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39f40ae2b0 0x7f39f409b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39fb2c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39f413f860 0x7f39f409b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39fb2c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39f415d620 0x7f39f409d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39fb2c8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39f415da80 con 0x7f39f40ae2b0 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39fb2c8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f39f40b01c0 con 0x7f39f413f860 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39fb2c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f39f406b510 con 0x7f39f415d620 2024-03-29T04:07:17.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39f8863640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39f413f860 0x7f39f409b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39f8863640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39f413f860 0x7f39f409b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36832/0 (socket says 172.21.15.154:36832) 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39f8863640 1 -- 172.21.15.154:0/3304367998 learned_addr learned my addr 172.21.15.154:0/3304367998 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39f9865640 1 --2- 172.21.15.154:0/3304367998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39f415d620 0x7f39f409d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39f9064640 1 --2- 172.21.15.154:0/3304367998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39f40ae2b0 0x7f39f409b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39f9865640 1 -- 172.21.15.154:0/3304367998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39f413f860 msgr2=0x7f39f409b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39f9865640 1 --2- 172.21.15.154:0/3304367998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39f413f860 0x7f39f409b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39f9865640 1 -- 172.21.15.154:0/3304367998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39f40ae2b0 msgr2=0x7f39f409b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39f9865640 1 --2- 172.21.15.154:0/3304367998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39f40ae2b0 0x7f39f409b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39f9865640 1 -- 172.21.15.154:0/3304367998 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f409c0e0 con 0x7f39f415d620 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39f8863640 1 --2- 172.21.15.154:0/3304367998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39f413f860 0x7f39f409b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39f9865640 1 --2- 172.21.15.154:0/3304367998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39f415d620 0x7f39f409d450 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00c6c0 tx=0x7f39ec00cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39e9ffb640 1 -- 172.21.15.154:0/3304367998 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec018070 con 0x7f39f415d620 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39e9ffb640 1 -- 172.21.15.154:0/3304367998 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39ec004d60 con 0x7f39f415d620 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.937+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3304367998 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f41e4ef0 con 0x7f39f415d620 2024-03-29T04:07:17.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.938+0000 7f39e9ffb640 1 -- 172.21.15.154:0/3304367998 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec00a840 con 0x7f39f415d620 2024-03-29T04:07:17.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.938+0000 7f39e9ffb640 1 -- 172.21.15.154:0/3304367998 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f39ec01b070 con 0x7f39f415d620 2024-03-29T04:07:17.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.938+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3304367998 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f413cc60 con 0x7f39f415d620 2024-03-29T04:07:17.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.938+0000 7f39e9ffb640 1 -- 172.21.15.154:0/3304367998 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f39ec0078b0 con 0x7f39f415d620 2024-03-29T04:07:17.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.939+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3304367998 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f409f0f0 con 0x7f39f415d620 2024-03-29T04:07:17.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:17.941+0000 7f39e9ffb640 1 -- 172.21.15.154:0/3304367998 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f39ec022020 con 0x7f39f415d620 2024-03-29T04:07:17.941 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:17.961+0000 7f659be1a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-29T04:07:17.954 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:17.974+0000 7f083402b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-29T04:07:17.988 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:18.009+0000 7f659be1a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-29T04:07:17.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.014+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3304367998 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1910d100-2ee1-4a4c-8eb5-427373f8d0c7", "id": 4} v 0) -- 0x7f39f413d740 con 0x7f39f415d620 2024-03-29T04:07:17.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.019+0000 7f39e9ffb640 1 -- 172.21.15.154:0/3304367998 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1910d100-2ee1-4a4c-8eb5-427373f8d0c7", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f39ec007070 con 0x7f39f415d620 2024-03-29T04:07:17.999 INFO:teuthology.orchestra.run.smithi154.stdout:4 2024-03-29T04:07:18.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.020+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3304367998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39f415d620 msgr2=0x7f39f409d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.020+0000 7f39fb2c8640 1 --2- 172.21.15.154:0/3304367998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39f415d620 0x7f39f409d450 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00c6c0 tx=0x7f39ec00cb80 comp rx=0 tx=0).stop 2024-03-29T04:07:18.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.020+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3304367998 shutdown_connections 2024-03-29T04:07:18.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.020+0000 7f39fb2c8640 1 --2- 172.21.15.154:0/3304367998 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f39f415d620 0x7f39f409d450 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.020+0000 7f39fb2c8640 1 --2- 172.21.15.154:0/3304367998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39f413f860 0x7f39f409b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.020+0000 7f39fb2c8640 1 --2- 172.21.15.154:0/3304367998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39f40ae2b0 0x7f39f409b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.020+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3304367998 >> 172.21.15.154:0/3304367998 conn(0x7f39f40d0bb0 msgr2=0x7f39f413de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:18.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.020+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3304367998 shutdown_connections 2024-03-29T04:07:18.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.020+0000 7f39fb2c8640 1 -- 172.21.15.154:0/3304367998 wait complete. 2024-03-29T04:07:18.006 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:18.026+0000 7f083402b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-29T04:07:18.010 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new a15d2654-a296-4f18-8db2-e3de9ff567f0 5 2024-03-29T04:07:18.045 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:18.066+0000 7f659be1a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-29T04:07:18.053 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:18.073+0000 7f083402b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-29T04:07:18.094 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:18.114+0000 7f659be1a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-29T04:07:18.111 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.131+0000 7f7eeb38d640 1 Processor -- start 2024-03-29T04:07:18.111 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.131+0000 7f7eeb38d640 1 -- start start 2024-03-29T04:07:18.112 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.132+0000 7f7eeb38d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ee40ce420 0x7f7ee40cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.112 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.132+0000 7f7eeb38d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ee4070560 0x7f7ee40cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.112 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.132+0000 7f7eeb38d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ee40cd780 con 0x7f7ee415b710 2024-03-29T04:07:18.112 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.132+0000 7f7eeb38d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee40cda60 con 0x7f7ee4070560 2024-03-29T04:07:18.112 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.132+0000 7f7eeb38d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee406ea70 con 0x7f7ee40ce420 2024-03-29T04:07:18.112 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.132+0000 7f7ee8928640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ee40ce420 0x7f7ee40cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.112 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.132+0000 7f7ee8928640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ee40ce420 0x7f7ee40cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36838/0 (socket says 172.21.15.154:36838) 2024-03-29T04:07:18.112 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.132+0000 7f7ee8928640 1 -- 172.21.15.154:0/3749328659 learned_addr learned my addr 172.21.15.154:0/3749328659 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:18.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.132+0000 7f7ee992a640 1 --2- 172.21.15.154:0/3749328659 >> v2:172.21.15.97:3300/0 conn(0x7f7ee4070560 0x7f7ee40cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.132+0000 7f7ee8928640 1 -- 172.21.15.154:0/3749328659 >> v2:172.21.15.97:3300/0 conn(0x7f7ee4070560 msgr2=0x7f7ee40cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:18.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.132+0000 7f7ee8928640 1 --2- 172.21.15.154:0/3749328659 >> v2:172.21.15.97:3300/0 conn(0x7f7ee4070560 0x7f7ee40cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.132+0000 7f7ee8928640 1 -- 172.21.15.154:0/3749328659 >> v1:172.21.15.97:6789/0 conn(0x7f7ee415b710 legacy=0x7f7ee40cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.132+0000 7f7ee8928640 1 -- 172.21.15.154:0/3749328659 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee406ed50 con 0x7f7ee40ce420 2024-03-29T04:07:18.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.133+0000 7f7ee8928640 1 --2- 172.21.15.154:0/3749328659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ee40ce420 0x7f7ee40cbb60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40b7490 tx=0x7f7ee002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82ae307a692ac2da server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:18.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.133+0000 7f7edbfff640 1 -- 172.21.15.154:0/3749328659 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0034070 con 0x7f7ee40ce420 2024-03-29T04:07:18.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.133+0000 7f7edbfff640 1 -- 172.21.15.154:0/3749328659 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee0030860 con 0x7f7ee40ce420 2024-03-29T04:07:18.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.133+0000 7f7edbfff640 1 -- 172.21.15.154:0/3749328659 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0030c30 con 0x7f7ee40ce420 2024-03-29T04:07:18.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.134+0000 7f7eeb38d640 1 -- 172.21.15.154:0/3749328659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ee40ce420 msgr2=0x7f7ee40cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.134+0000 7f7eeb38d640 1 --2- 172.21.15.154:0/3749328659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ee40ce420 0x7f7ee40cbb60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40b7490 tx=0x7f7ee002b9a0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.134+0000 7f7eeb38d640 1 -- 172.21.15.154:0/3749328659 shutdown_connections 2024-03-29T04:07:18.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.134+0000 7f7eeb38d640 1 --2- 172.21.15.154:0/3749328659 >> v2:172.21.15.97:3300/0 conn(0x7f7ee4070560 0x7f7ee40cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.134+0000 7f7eeb38d640 1 --2- 172.21.15.154:0/3749328659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ee40ce420 0x7f7ee40cbb60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.134+0000 7f7eeb38d640 1 -- 172.21.15.154:0/3749328659 >> 172.21.15.154:0/3749328659 conn(0x7f7ee40b2ed0 msgr2=0x7f7ee40b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:18.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.134+0000 7f7eeb38d640 1 -- 172.21.15.154:0/3749328659 shutdown_connections 2024-03-29T04:07:18.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.134+0000 7f7eeb38d640 1 -- 172.21.15.154:0/3749328659 wait complete. 2024-03-29T04:07:18.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.135+0000 7f7eeb38d640 1 Processor -- start 2024-03-29T04:07:18.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.135+0000 7f7eeb38d640 1 -- start start 2024-03-29T04:07:18.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7eeb38d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee4070560 0x7f7ee406b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7eeb38d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ee40ce420 0x7f7ee406ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7eeb38d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ee415b710 0x7f7ee40c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7eeb38d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee40b3d20 con 0x7f7ee4070560 2024-03-29T04:07:18.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7eeb38d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee40b3ba0 con 0x7f7ee40ce420 2024-03-29T04:07:18.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7eeb38d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee415baf0 con 0x7f7ee415b710 2024-03-29T04:07:18.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7ee8928640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ee40ce420 0x7f7ee406ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7ee992a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ee415b710 0x7f7ee40c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7ee8928640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ee40ce420 0x7f7ee406ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36850/0 (socket says 172.21.15.154:36850) 2024-03-29T04:07:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7ee8928640 1 -- 172.21.15.154:0/760046636 learned_addr learned my addr 172.21.15.154:0/760046636 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7ee9129640 1 --2- 172.21.15.154:0/760046636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee4070560 0x7f7ee406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7ee992a640 1 -- 172.21.15.154:0/760046636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ee40ce420 msgr2=0x7f7ee406ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7ee992a640 1 --2- 172.21.15.154:0/760046636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ee40ce420 0x7f7ee406ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7ee992a640 1 -- 172.21.15.154:0/760046636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee4070560 msgr2=0x7f7ee406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7ee992a640 1 --2- 172.21.15.154:0/760046636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee4070560 0x7f7ee406b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7ee992a640 1 -- 172.21.15.154:0/760046636 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee409b3d0 con 0x7f7ee415b710 2024-03-29T04:07:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7ee8928640 1 --2- 172.21.15.154:0/760046636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ee40ce420 0x7f7ee406ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.136+0000 7f7ee992a640 1 --2- 172.21.15.154:0/760046636 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ee415b710 0x7f7ee40c9420 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00efc0 tx=0x7f7edc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.137+0000 7f7ed9ffb640 1 -- 172.21.15.154:0/760046636 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc018020 con 0x7f7ee415b710 2024-03-29T04:07:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.137+0000 7f7ed9ffb640 1 -- 172.21.15.154:0/760046636 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7edc014480 con 0x7f7ee415b710 2024-03-29T04:07:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.137+0000 7f7eeb38d640 1 -- 172.21.15.154:0/760046636 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee409b640 con 0x7f7ee415b710 2024-03-29T04:07:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.137+0000 7f7ed9ffb640 1 -- 172.21.15.154:0/760046636 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc014870 con 0x7f7ee415b710 2024-03-29T04:07:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.137+0000 7f7eeb38d640 1 -- 172.21.15.154:0/760046636 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee409bc00 con 0x7f7ee415b710 2024-03-29T04:07:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.137+0000 7f7ed9ffb640 1 -- 172.21.15.154:0/760046636 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7edc005d10 con 0x7f7ee415b710 2024-03-29T04:07:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.137+0000 7f7ed9ffb640 1 -- 172.21.15.154:0/760046636 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f7edc013960 con 0x7f7ee415b710 2024-03-29T04:07:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.138+0000 7f7eeb38d640 1 -- 172.21.15.154:0/760046636 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee409f530 con 0x7f7ee415b710 2024-03-29T04:07:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.140+0000 7f7ed9ffb640 1 -- 172.21.15.154:0/760046636 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7edc022030 con 0x7f7ee415b710 2024-03-29T04:07:18.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.212+0000 7f7eeb38d640 1 -- 172.21.15.154:0/760046636 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a15d2654-a296-4f18-8db2-e3de9ff567f0", "id": 5} v 0) -- 0x7f7ee404f9d0 con 0x7f7ee415b710 2024-03-29T04:07:18.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.218+0000 7f7ed9ffb640 1 -- 172.21.15.154:0/760046636 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a15d2654-a296-4f18-8db2-e3de9ff567f0", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f7edc010040 con 0x7f7ee415b710 2024-03-29T04:07:18.197 INFO:teuthology.orchestra.run.smithi154.stdout:5 2024-03-29T04:07:18.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.219+0000 7f7eeb38d640 1 -- 172.21.15.154:0/760046636 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ee415b710 msgr2=0x7f7ee40c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.219+0000 7f7eeb38d640 1 --2- 172.21.15.154:0/760046636 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ee415b710 0x7f7ee40c9420 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00efc0 tx=0x7f7edc00c490 comp rx=0 tx=0).stop 2024-03-29T04:07:18.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.219+0000 7f7eeb38d640 1 -- 172.21.15.154:0/760046636 shutdown_connections 2024-03-29T04:07:18.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.219+0000 7f7eeb38d640 1 --2- 172.21.15.154:0/760046636 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ee415b710 0x7f7ee40c9420 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.219+0000 7f7eeb38d640 1 --2- 172.21.15.154:0/760046636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ee40ce420 0x7f7ee406ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.219+0000 7f7eeb38d640 1 --2- 172.21.15.154:0/760046636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee4070560 0x7f7ee406b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.219+0000 7f7eeb38d640 1 -- 172.21.15.154:0/760046636 >> 172.21.15.154:0/760046636 conn(0x7f7ee40b2ed0 msgr2=0x7f7ee40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:18.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.219+0000 7f7eeb38d640 1 -- 172.21.15.154:0/760046636 shutdown_connections 2024-03-29T04:07:18.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.219+0000 7f7eeb38d640 1 -- 172.21.15.154:0/760046636 wait complete. 2024-03-29T04:07:18.209 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef 6 2024-03-29T04:07:18.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.371+0000 7f1c1d270640 1 Processor -- start 2024-03-29T04:07:18.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.371+0000 7f1c1d270640 1 -- start start 2024-03-29T04:07:18.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.372+0000 7f1c1d270640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1c180b3dc0 0x7f1c180b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.372+0000 7f1c1d270640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c18067960 con 0x7f1c180cf0a0 2024-03-29T04:07:18.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.372+0000 7f1c1d270640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c1806c9d0 con 0x7f1c180b48b0 2024-03-29T04:07:18.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.372+0000 7f1c1d270640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f1c1806ee40 con 0x7f1c180b3dc0 2024-03-29T04:07:18.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.372+0000 7f1c16ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1c180b3dc0 0x7f1c180b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.372+0000 7f1c16ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1c180b3dc0 0x7f1c180b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36860/0 (socket says 172.21.15.154:36860) 2024-03-29T04:07:18.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.372+0000 7f1c16ffd640 1 -- 172.21.15.154:0/3593998484 learned_addr learned my addr 172.21.15.154:0/3593998484 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:18.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.373+0000 7f1c16ffd640 1 -- 172.21.15.154:0/3593998484 >> v1:172.21.15.154:6789/0 conn(0x7f1c180b48b0 legacy=0x7f1c18165140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.373+0000 7f1c16ffd640 1 -- 172.21.15.154:0/3593998484 >> v1:172.21.15.97:6789/0 conn(0x7f1c180cf0a0 legacy=0x7f1c180b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.373+0000 7f1c16ffd640 1 -- 172.21.15.154:0/3593998484 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c18067680 con 0x7f1c180b3dc0 2024-03-29T04:07:18.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.373+0000 7f1c16ffd640 1 --2- 172.21.15.154:0/3593998484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c180b3dc0 0x7f1c180b5a50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009870 tx=0x7f1c0002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f1179436c1d06b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:18.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.374+0000 7f1c15ffb640 1 -- 172.21.15.154:0/3593998484 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00034070 con 0x7f1c180b3dc0 2024-03-29T04:07:18.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.374+0000 7f1c15ffb640 1 -- 172.21.15.154:0/3593998484 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c00030860 con 0x7f1c180b3dc0 2024-03-29T04:07:18.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.374+0000 7f1c15ffb640 1 -- 172.21.15.154:0/3593998484 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00030c30 con 0x7f1c180b3dc0 2024-03-29T04:07:18.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.374+0000 7f1c1d270640 1 -- 172.21.15.154:0/3593998484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c180b3dc0 msgr2=0x7f1c180b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.374+0000 7f1c1d270640 1 --2- 172.21.15.154:0/3593998484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c180b3dc0 0x7f1c180b5a50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009870 tx=0x7f1c0002b9a0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.374+0000 7f1c1d270640 1 -- 172.21.15.154:0/3593998484 shutdown_connections 2024-03-29T04:07:18.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.374+0000 7f1c1d270640 1 --2- 172.21.15.154:0/3593998484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c180b3dc0 0x7f1c180b5a50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.374+0000 7f1c1d270640 1 -- 172.21.15.154:0/3593998484 >> 172.21.15.154:0/3593998484 conn(0x7f1c180d0ac0 msgr2=0x7f1c180d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:18.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.374+0000 7f1c1d270640 1 -- 172.21.15.154:0/3593998484 shutdown_connections 2024-03-29T04:07:18.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.374+0000 7f1c1d270640 1 -- 172.21.15.154:0/3593998484 wait complete. 2024-03-29T04:07:18.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.376+0000 7f1c1d270640 1 Processor -- start 2024-03-29T04:07:18.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.376+0000 7f1c1d270640 1 -- start start 2024-03-29T04:07:18.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.376+0000 7f1c1d270640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c180b3dc0 0x7f1c18142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.376+0000 7f1c1d270640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c180b48b0 0x7f1c181e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.376+0000 7f1c1d270640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c180cf0a0 0x7f1c181e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.376+0000 7f1c1d270640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1c180cecf0 con 0x7f1c180cf0a0 2024-03-29T04:07:18.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.376+0000 7f1c1d270640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18067330 con 0x7f1c180b48b0 2024-03-29T04:07:18.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.376+0000 7f1c1d270640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1c181e7600 con 0x7f1c180b3dc0 2024-03-29T04:07:18.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c16ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c180b3dc0 0x7f1c18142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c16ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c180b3dc0 0x7f1c18142770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:57646/0 (socket says 172.21.15.154:57646) 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c16ffd640 1 -- 172.21.15.154:0/915769485 learned_addr learned my addr 172.21.15.154:0/915769485 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c167fc640 1 --2- 172.21.15.154:0/915769485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c180b48b0 0x7f1c181e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c177fe640 1 --2- 172.21.15.154:0/915769485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c180cf0a0 0x7f1c181e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c16ffd640 1 -- 172.21.15.154:0/915769485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c180b48b0 msgr2=0x7f1c181e7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c16ffd640 1 --2- 172.21.15.154:0/915769485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c180b48b0 0x7f1c181e7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c16ffd640 1 -- 172.21.15.154:0/915769485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c180cf0a0 msgr2=0x7f1c181e83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c16ffd640 1 --2- 172.21.15.154:0/915769485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c180cf0a0 0x7f1c181e83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c16ffd640 1 -- 172.21.15.154:0/915769485 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c181e3f30 con 0x7f1c180b3dc0 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c167fc640 1 --2- 172.21.15.154:0/915769485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c180b48b0 0x7f1c181e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c16ffd640 1 --2- 172.21.15.154:0/915769485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c180b3dc0 0x7f1c18142770 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1c00000c00 tx=0x7f1c000046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c0f7fe640 1 -- 172.21.15.154:0/915769485 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00034040 con 0x7f1c180b3dc0 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c0f7fe640 1 -- 172.21.15.154:0/915769485 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c000433f0 con 0x7f1c180b3dc0 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c1d270640 1 -- 172.21.15.154:0/915769485 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c181e4210 con 0x7f1c180b3dc0 2024-03-29T04:07:18.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c0f7fe640 1 -- 172.21.15.154:0/915769485 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c000437e0 con 0x7f1c180b3dc0 2024-03-29T04:07:18.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.377+0000 7f1c1d270640 1 -- 172.21.15.154:0/915769485 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c181e6910 con 0x7f1c180b3dc0 2024-03-29T04:07:18.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.378+0000 7f1c0f7fe640 1 -- 172.21.15.154:0/915769485 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1c0004a060 con 0x7f1c180b3dc0 2024-03-29T04:07:18.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.378+0000 7f1c0f7fe640 1 -- 172.21.15.154:0/915769485 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f1c0004cbd0 con 0x7f1c180b3dc0 2024-03-29T04:07:18.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.378+0000 7f1c1d270640 1 -- 172.21.15.154:0/915769485 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c18073af0 con 0x7f1c180b3dc0 2024-03-29T04:07:18.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.380+0000 7f1c0f7fe640 1 -- 172.21.15.154:0/915769485 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1c0003e070 con 0x7f1c180b3dc0 2024-03-29T04:07:18.428 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:18.448+0000 7f083402b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-29T04:07:18.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.453+0000 7f1c1d270640 1 -- 172.21.15.154:0/915769485 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef", "id": 6} v 0) -- 0x7f1c180cee70 con 0x7f1c180b3dc0 2024-03-29T04:07:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.458+0000 7f1c0f7fe640 1 -- 172.21.15.154:0/915769485 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f1c00030720 con 0x7f1c180b3dc0 2024-03-29T04:07:18.437 INFO:teuthology.orchestra.run.smithi154.stdout:6 2024-03-29T04:07:18.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.459+0000 7f1c1d270640 1 -- 172.21.15.154:0/915769485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c180b3dc0 msgr2=0x7f1c18142770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.459+0000 7f1c1d270640 1 --2- 172.21.15.154:0/915769485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c180b3dc0 0x7f1c18142770 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1c00000c00 tx=0x7f1c000046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.459+0000 7f1c1d270640 1 -- 172.21.15.154:0/915769485 shutdown_connections 2024-03-29T04:07:18.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.459+0000 7f1c1d270640 1 --2- 172.21.15.154:0/915769485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c180cf0a0 0x7f1c181e83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.459+0000 7f1c1d270640 1 --2- 172.21.15.154:0/915769485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c180b48b0 0x7f1c181e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.459+0000 7f1c1d270640 1 --2- 172.21.15.154:0/915769485 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1c180b3dc0 0x7f1c18142770 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.459+0000 7f1c1d270640 1 -- 172.21.15.154:0/915769485 >> 172.21.15.154:0/915769485 conn(0x7f1c180d0ac0 msgr2=0x7f1c18071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:18.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.459+0000 7f1c1d270640 1 -- 172.21.15.154:0/915769485 shutdown_connections 2024-03-29T04:07:18.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.459+0000 7f1c1d270640 1 -- 172.21.15.154:0/915769485 wait complete. 2024-03-29T04:07:18.449 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new a9236f6d-34f1-4ace-a310-b63fad918bc1 7 2024-03-29T04:07:18.497 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:18.518+0000 7f659be1a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-29T04:07:18.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.571+0000 7fb6d5500640 1 Processor -- start 2024-03-29T04:07:18.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.571+0000 7fb6d5500640 1 -- start start 2024-03-29T04:07:18.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6d5500640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6d00cd710 0x7fb6d00b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6d5500640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6d006ea70 con 0x7fb6d006bc20 2024-03-29T04:07:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6d5500640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6d00d0800 con 0x7fb6d015ea00 2024-03-29T04:07:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6d5500640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fb6d00d0ae0 con 0x7fb6d00cd710 2024-03-29T04:07:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6ceffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb6d015ea00 0x7fb6d006b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47034/0 (socket says 172.21.15.154:47034) 2024-03-29T04:07:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6ceffd640 1 -- 172.21.15.154:0/697716932 learned_addr learned my addr 172.21.15.154:0/697716932 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-29T04:07:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6cf7fe640 1 --2- 172.21.15.154:0/697716932 >> v2:172.21.15.97:3300/0 conn(0x7fb6d00cd710 0x7fb6d00b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6ce7fc640 1 -- 172.21.15.154:0/697716932 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2667053453 0 0) 0x7fb6d00d0800 con 0x7fb6d015ea00 2024-03-29T04:07:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6ce7fc640 1 -- 172.21.15.154:0/697716932 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6bc003520 con 0x7fb6d015ea00 2024-03-29T04:07:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6ce7fc640 1 -- 172.21.15.154:0/697716932 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3265663500 0 0) 0x7fb6d006ea70 con 0x7fb6d006bc20 2024-03-29T04:07:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6ce7fc640 1 -- 172.21.15.154:0/697716932 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6d00d0800 con 0x7fb6d006bc20 2024-03-29T04:07:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6cf7fe640 1 -- 172.21.15.154:0/697716932 >> v1:172.21.15.154:6789/0 conn(0x7fb6d015ea00 legacy=0x7fb6d006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6ce7fc640 1 -- 172.21.15.154:0/697716932 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4207265717 0 0) 0x7fb6bc003520 con 0x7fb6d015ea00 2024-03-29T04:07:18.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6cf7fe640 1 -- 172.21.15.154:0/697716932 >> v1:172.21.15.97:6789/0 conn(0x7fb6d006bc20 legacy=0x7fb6d006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.572+0000 7fb6cf7fe640 1 -- 172.21.15.154:0/697716932 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d00d0dc0 con 0x7fb6d00cd710 2024-03-29T04:07:18.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.573+0000 7fb6cf7fe640 1 --2- 172.21.15.154:0/697716932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d00cd710 0x7fb6d00b0600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009d90 tx=0x7fb6c802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f291262c68270a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:18.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.573+0000 7fb6ce7fc640 1 -- 172.21.15.154:0/697716932 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8039070 con 0x7fb6d00cd710 2024-03-29T04:07:18.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.573+0000 7fb6ce7fc640 1 -- 172.21.15.154:0/697716932 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c802c970 con 0x7fb6d00cd710 2024-03-29T04:07:18.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.573+0000 7fb6ce7fc640 1 -- 172.21.15.154:0/697716932 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0001670 con 0x7fb6d00cd710 2024-03-29T04:07:18.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.574+0000 7fb6d5500640 1 -- 172.21.15.154:0/697716932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d00cd710 msgr2=0x7fb6d00b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.574+0000 7fb6d5500640 1 --2- 172.21.15.154:0/697716932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d00cd710 0x7fb6d00b0600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009d90 tx=0x7fb6c802be40 comp rx=0 tx=0).stop 2024-03-29T04:07:18.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.574+0000 7fb6d5500640 1 -- 172.21.15.154:0/697716932 shutdown_connections 2024-03-29T04:07:18.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.574+0000 7fb6d5500640 1 --2- 172.21.15.154:0/697716932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d00cd710 0x7fb6d00b0600 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.574+0000 7fb6d5500640 1 -- 172.21.15.154:0/697716932 >> 172.21.15.154:0/697716932 conn(0x7fb6d00b2ed0 msgr2=0x7fb6d00b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:18.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.574+0000 7fb6d5500640 1 -- 172.21.15.154:0/697716932 shutdown_connections 2024-03-29T04:07:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.574+0000 7fb6d5500640 1 -- 172.21.15.154:0/697716932 wait complete. 2024-03-29T04:07:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.575+0000 7fb6d5500640 1 Processor -- start 2024-03-29T04:07:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.575+0000 7fb6d5500640 1 -- start start 2024-03-29T04:07:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6d5500640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6d006bc20 0x7fb6d00c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6d5500640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d00cd710 0x7fb6d014e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6d5500640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d015ea00 0x7fb6d00b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6d5500640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d015ede0 con 0x7fb6d015ea00 2024-03-29T04:07:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6d5500640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d00d04b0 con 0x7fb6d00cd710 2024-03-29T04:07:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6d5500640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d015ef60 con 0x7fb6d006bc20 2024-03-29T04:07:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6c7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d00cd710 0x7fb6d014e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6c7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d00cd710 0x7fb6d014e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36878/0 (socket says 172.21.15.154:36878) 2024-03-29T04:07:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6c7fff640 1 -- 172.21.15.154:0/1393151828 learned_addr learned my addr 172.21.15.154:0/1393151828 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-29T04:07:18.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6ceffd640 1 --2- 172.21.15.154:0/1393151828 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6d006bc20 0x7fb6d00c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6cf7fe640 1 --2- 172.21.15.154:0/1393151828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d015ea00 0x7fb6d00b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6c7fff640 1 -- 172.21.15.154:0/1393151828 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6d006bc20 msgr2=0x7fb6d00c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6c7fff640 1 --2- 172.21.15.154:0/1393151828 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6d006bc20 0x7fb6d00c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6c7fff640 1 -- 172.21.15.154:0/1393151828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d015ea00 msgr2=0x7fb6d00b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6c7fff640 1 --2- 172.21.15.154:0/1393151828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d015ea00 0x7fb6d00b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6c7fff640 1 -- 172.21.15.154:0/1393151828 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d00cb6c0 con 0x7fb6d00cd710 2024-03-29T04:07:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.576+0000 7fb6c7fff640 1 --2- 172.21.15.154:0/1393151828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d00cd710 0x7fb6d014e640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800c490 tx=0x7fb6b800c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.577+0000 7fb6c77fe640 1 -- 172.21.15.154:0/1393151828 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b80052d0 con 0x7fb6d00cd710 2024-03-29T04:07:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.577+0000 7fb6c77fe640 1 -- 172.21.15.154:0/1393151828 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b8005470 con 0x7fb6d00cd710 2024-03-29T04:07:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.577+0000 7fb6c77fe640 1 -- 172.21.15.154:0/1393151828 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8012850 con 0x7fb6d00cd710 2024-03-29T04:07:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.577+0000 7fb6d5500640 1 -- 172.21.15.154:0/1393151828 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d00cb3e0 con 0x7fb6d00cd710 2024-03-29T04:07:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.577+0000 7fb6d5500640 1 -- 172.21.15.154:0/1393151828 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d01e8ac0 con 0x7fb6d00cd710 2024-03-29T04:07:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.577+0000 7fb6c77fe640 1 -- 172.21.15.154:0/1393151828 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb6b80129f0 con 0x7fb6d00cd710 2024-03-29T04:07:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.577+0000 7fb6c77fe640 1 -- 172.21.15.154:0/1393151828 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fb6b801bb50 con 0x7fb6d00cd710 2024-03-29T04:07:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.578+0000 7fb6d5500640 1 -- 172.21.15.154:0/1393151828 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d00706b0 con 0x7fb6d00cd710 2024-03-29T04:07:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.580+0000 7fb6c77fe640 1 -- 172.21.15.154:0/1393151828 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb6b801b4f0 con 0x7fb6d00cd710 2024-03-29T04:07:18.563 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:18.583+0000 7f083402b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-29T04:07:18.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.652+0000 7fb6d5500640 1 -- 172.21.15.154:0/1393151828 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a9236f6d-34f1-4ace-a310-b63fad918bc1", "id": 7} v 0) -- 0x7fb6d004f9d0 con 0x7fb6d00cd710 2024-03-29T04:07:18.639 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:18.658+0000 7f659be1a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-29T04:07:18.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.658+0000 7fb6c77fe640 1 -- 172.21.15.154:0/1393151828 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a9236f6d-34f1-4ace-a310-b63fad918bc1", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fb6b801b6d0 con 0x7fb6d00cd710 2024-03-29T04:07:18.640 INFO:teuthology.orchestra.run.smithi154.stdout:7 2024-03-29T04:07:18.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.659+0000 7fb6d5500640 1 -- 172.21.15.154:0/1393151828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d00cd710 msgr2=0x7fb6d014e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.659+0000 7fb6d5500640 1 --2- 172.21.15.154:0/1393151828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d00cd710 0x7fb6d014e640 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800c490 tx=0x7fb6b800c950 comp rx=0 tx=0).stop 2024-03-29T04:07:18.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.659+0000 7fb6d5500640 1 -- 172.21.15.154:0/1393151828 shutdown_connections 2024-03-29T04:07:18.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.659+0000 7fb6d5500640 1 --2- 172.21.15.154:0/1393151828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d015ea00 0x7fb6d00b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.659+0000 7fb6d5500640 1 --2- 172.21.15.154:0/1393151828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6d00cd710 0x7fb6d014e640 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.659+0000 7fb6d5500640 1 --2- 172.21.15.154:0/1393151828 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6d006bc20 0x7fb6d00c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.659+0000 7fb6d5500640 1 -- 172.21.15.154:0/1393151828 >> 172.21.15.154:0/1393151828 conn(0x7fb6d00b2ed0 msgr2=0x7fb6d006f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:18.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.659+0000 7fb6d5500640 1 -- 172.21.15.154:0/1393151828 shutdown_connections 2024-03-29T04:07:18.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:18.659+0000 7fb6d5500640 1 -- 172.21.15.154:0/1393151828 wait complete. 2024-03-29T04:07:18.651 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-29T04:07:18.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-29T04:07:18.655 INFO:tasks.ceph.osd.0:Started 2024-03-29T04:07:18.655 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-29T04:07:18.656 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-29T04:07:18.665 INFO:tasks.ceph.osd.1:Started 2024-03-29T04:07:18.666 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-29T04:07:18.666 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-29T04:07:18.668 INFO:tasks.ceph.osd.2:Started 2024-03-29T04:07:18.669 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-29T04:07:18.669 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-29T04:07:18.671 INFO:tasks.ceph.osd.3:Started 2024-03-29T04:07:18.672 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-29T04:07:18.672 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-29T04:07:18.675 INFO:tasks.ceph.osd.4:Started 2024-03-29T04:07:18.675 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-29T04:07:18.675 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-29T04:07:18.682 INFO:tasks.ceph.osd.5:Started 2024-03-29T04:07:18.682 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-29T04:07:18.682 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-29T04:07:18.685 INFO:tasks.ceph.osd.6:Started 2024-03-29T04:07:18.685 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-29T04:07:18.685 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-29T04:07:18.688 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:18.708+0000 7f083402b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-29T04:07:18.688 INFO:tasks.ceph.osd.7:Started 2024-03-29T04:07:18.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:18.762 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:18.782+0000 7f083402b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-29T04:07:18.783 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:18.804+0000 7f659be1a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-29T04:07:18.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.830+0000 7fb6bd8eb640 1 Processor -- start 2024-03-29T04:07:18.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.830+0000 7fb6bd8eb640 1 -- start start 2024-03-29T04:07:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.831+0000 7fb6bd8eb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6b81447a0 0x7fb6b80b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.831+0000 7fb6bd8eb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b806cc30 0x7fb6b806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.831+0000 7fb6bd8eb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6b806e150 con 0x7fb6b80b4c60 2024-03-29T04:07:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.831+0000 7fb6bd8eb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b80cd9e0 con 0x7fb6b81447a0 2024-03-29T04:07:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.831+0000 7fb6bd8eb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b806e730 con 0x7fb6b806cc30 2024-03-29T04:07:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.831+0000 7fb6b6ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6b81447a0 0x7fb6b80b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.831+0000 7fb6b6ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6b81447a0 0x7fb6b80b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49676/0 (socket says 172.21.15.97:49676) 2024-03-29T04:07:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.831+0000 7fb6b6ffd640 1 -- 172.21.15.97:0/46964772 learned_addr learned my addr 172.21.15.97:0/46964772 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.831+0000 7fb6b77fe640 1 --2- 172.21.15.97:0/46964772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b806cc30 0x7fb6b806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.831+0000 7fb6b6ffd640 1 -- 172.21.15.97:0/46964772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b806cc30 msgr2=0x7fb6b806ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.831+0000 7fb6b6ffd640 1 --2- 172.21.15.97:0/46964772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b806cc30 0x7fb6b806ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.831+0000 7fb6b6ffd640 1 -- 172.21.15.97:0/46964772 >> v1:172.21.15.154:6789/0 conn(0x7fb6b80b4c60 legacy=0x7fb6b806c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.831+0000 7fb6b6ffd640 1 -- 172.21.15.97:0/46964772 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b80b3da0 con 0x7fb6b81447a0 2024-03-29T04:07:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.832+0000 7fb6b6ffd640 1 --2- 172.21.15.97:0/46964772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b81447a0 0x7fb6b80b4720 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009870 tx=0x7fb6a802ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145d6560ade005ef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.832+0000 7fb6b5ffb640 1 -- 172.21.15.97:0/46964772 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a803b070 con 0x7fb6b81447a0 2024-03-29T04:07:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.832+0000 7fb6b5ffb640 1 -- 172.21.15.97:0/46964772 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a802eb70 con 0x7fb6b81447a0 2024-03-29T04:07:18.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.832+0000 7fb6b5ffb640 1 -- 172.21.15.97:0/46964772 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8036440 con 0x7fb6b81447a0 2024-03-29T04:07:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.833+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/46964772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b81447a0 msgr2=0x7fb6b80b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.833+0000 7fb6bd8eb640 1 --2- 172.21.15.97:0/46964772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b81447a0 0x7fb6b80b4720 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009870 tx=0x7fb6a802ddc0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.833+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/46964772 shutdown_connections 2024-03-29T04:07:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.833+0000 7fb6bd8eb640 1 --2- 172.21.15.97:0/46964772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b806cc30 0x7fb6b806ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.833+0000 7fb6bd8eb640 1 --2- 172.21.15.97:0/46964772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b81447a0 0x7fb6b80b4720 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.833+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/46964772 >> 172.21.15.97:0/46964772 conn(0x7fb6b8072050 msgr2=0x7fb6b8072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.833+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/46964772 shutdown_connections 2024-03-29T04:07:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.833+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/46964772 wait complete. 2024-03-29T04:07:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.834+0000 7fb6bd8eb640 1 Processor -- start 2024-03-29T04:07:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.834+0000 7fb6bd8eb640 1 -- start start 2024-03-29T04:07:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6bd8eb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b806cc30 0x7fb6b81f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6bd8eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b80b4c60 0x7fb6b80c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6bd8eb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b81447a0 0x7fb6b815d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6bd8eb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b80b5570 con 0x7fb6b80b4c60 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6bd8eb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b806e3c0 con 0x7fb6b81447a0 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6bd8eb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b80b56f0 con 0x7fb6b806cc30 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6b67fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b80b4c60 0x7fb6b80c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6b67fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b80b4c60 0x7fb6b80c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49692/0 (socket says 172.21.15.97:49692) 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6b67fc640 1 -- 172.21.15.97:0/2459393502 learned_addr learned my addr 172.21.15.97:0/2459393502 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6b77fe640 1 --2- 172.21.15.97:0/2459393502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b81447a0 0x7fb6b815d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6b67fc640 1 -- 172.21.15.97:0/2459393502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b806cc30 msgr2=0x7fb6b81f4ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6b67fc640 1 --2- 172.21.15.97:0/2459393502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b806cc30 0x7fb6b81f4ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6b67fc640 1 -- 172.21.15.97:0/2459393502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b81447a0 msgr2=0x7fb6b815d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6b67fc640 1 --2- 172.21.15.97:0/2459393502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b81447a0 0x7fb6b815d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6b67fc640 1 -- 172.21.15.97:0/2459393502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b80c6ac0 con 0x7fb6b80b4c60 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb6b67fc640 1 --2- 172.21.15.97:0/2459393502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b80b4c60 0x7fb6b80c6150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00c400 tx=0x7fb6ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:18.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb69ffff640 1 -- 172.21.15.97:0/2459393502 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0052d0 con 0x7fb6b80b4c60 2024-03-29T04:07:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.835+0000 7fb69ffff640 1 -- 172.21.15.97:0/2459393502 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ac005490 con 0x7fb6b80b4c60 2024-03-29T04:07:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.836+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/2459393502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b80c6da0 con 0x7fb6b80b4c60 2024-03-29T04:07:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.836+0000 7fb69ffff640 1 -- 172.21.15.97:0/2459393502 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac007260 con 0x7fb6b80b4c60 2024-03-29T04:07:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.836+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/2459393502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b80c7360 con 0x7fb6b80b4c60 2024-03-29T04:07:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.836+0000 7fb69ffff640 1 -- 172.21.15.97:0/2459393502 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb6ac007400 con 0x7fb6b80b4c60 2024-03-29T04:07:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.836+0000 7fb69ffff640 1 -- 172.21.15.97:0/2459393502 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fb6ac011c60 con 0x7fb6b80b4c60 2024-03-29T04:07:18.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.836+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/2459393502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b8068d50 con 0x7fb6b80b4c60 2024-03-29T04:07:18.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.839+0000 7fb69ffff640 1 -- 172.21.15.97:0/2459393502 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb6ac0035f0 con 0x7fb6b80b4c60 2024-03-29T04:07:18.830 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:18.850+0000 7f659be1a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-29T04:07:18.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.899+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/2459393502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6b8072d50 con 0x7fb6b80b4c60 2024-03-29T04:07:18.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.900+0000 7fb69ffff640 1 -- 172.21.15.97:0/2459393502 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fb6ac003a50 con 0x7fb6b80b4c60 2024-03-29T04:07:18.880 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:18.880 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:18.654511+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:18.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.901+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/2459393502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b80b4c60 msgr2=0x7fb6b80c6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:18.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.901+0000 7fb6bd8eb640 1 --2- 172.21.15.97:0/2459393502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b80b4c60 0x7fb6b80c6150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00c400 tx=0x7fb6ac00c8c0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.901+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/2459393502 shutdown_connections 2024-03-29T04:07:18.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.901+0000 7fb6bd8eb640 1 --2- 172.21.15.97:0/2459393502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6b81447a0 0x7fb6b815d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.901+0000 7fb6bd8eb640 1 --2- 172.21.15.97:0/2459393502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b80b4c60 0x7fb6b80c6150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.901+0000 7fb6bd8eb640 1 --2- 172.21.15.97:0/2459393502 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb6b806cc30 0x7fb6b81f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:18.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.901+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/2459393502 >> 172.21.15.97:0/2459393502 conn(0x7fb6b8072050 msgr2=0x7fb6b8143570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:18.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.901+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/2459393502 shutdown_connections 2024-03-29T04:07:18.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:18.901+0000 7fb6bd8eb640 1 -- 172.21.15.97:0/2459393502 wait complete. 2024-03-29T04:07:18.883 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:18.903+0000 7f083402b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:07:18.891 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-29T04:07:18.891 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-29T04:07:18.941 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:18.962+0000 7f659be1a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:07:18.980 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:19.001+0000 7f083402b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-29T04:07:19.038 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:19.058+0000 7f659be1a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-29T04:07:19.138 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:19.158+0000 7f083402b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:07:19.180 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:19.201+0000 7f083402b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-29T04:07:19.193 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:19.214+0000 7f659be1a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:07:19.237 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:19.257+0000 7f659be1a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-29T04:07:19.269 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-03-29T04:07:19.289+0000 7ff1e981b640 -1 Falling back to public interface 2024-03-29T04:07:19.270 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-03-29T04:07:19.290+0000 7f29cec21640 -1 Falling back to public interface 2024-03-29T04:07:19.282 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-03-29T04:07:19.302+0000 7f31ef448640 -1 Falling back to public interface 2024-03-29T04:07:19.284 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-03-29T04:07:19.304+0000 7f5c0661e640 -1 Falling back to public interface 2024-03-29T04:07:19.293 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-03-29T04:07:19.312+0000 7fbf2141f640 -1 Falling back to public interface 2024-03-29T04:07:19.294 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-03-29T04:07:19.314+0000 7fc6e0f2a640 -1 Falling back to public interface 2024-03-29T04:07:19.296 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-03-29T04:07:19.316+0000 7fc1a7814640 -1 Falling back to public interface 2024-03-29T04:07:19.301 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-03-29T04:07:19.320+0000 7f7abe0f7640 -1 Falling back to public interface 2024-03-29T04:07:19.339 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:19.359+0000 7f083402b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-29T04:07:19.394 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:19.415+0000 7f659be1a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-29T04:07:19.696 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:19.788 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.808+0000 7f87c5760640 1 Processor -- start 2024-03-29T04:07:19.788 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.808+0000 7f87c5760640 1 -- start start 2024-03-29T04:07:19.789 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.809+0000 7f87c5760640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f87c00b8260 0x7f87c00d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:19.789 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.809+0000 7f87c5760640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87c00d3d00 con 0x7f87c015eae0 2024-03-29T04:07:19.789 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.809+0000 7f87c5760640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87c00cfd80 con 0x7f87c00d1630 2024-03-29T04:07:19.789 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.809+0000 7f87c5760640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f87c00d1fd0 con 0x7f87c00b8260 2024-03-29T04:07:19.789 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.809+0000 7f87beffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f87c015eae0 0x7f87c00d35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:45398/0 (socket says 172.21.15.97:45398) 2024-03-29T04:07:19.790 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.809+0000 7f87beffd640 1 -- 172.21.15.97:0/30464658 learned_addr learned my addr 172.21.15.97:0/30464658 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:19.790 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.809+0000 7f87bf7fe640 1 --2- 172.21.15.97:0/30464658 >> v2:172.21.15.154:3300/0 conn(0x7f87c00b8260 0x7f87c00d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:19.790 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.809+0000 7f87be7fc640 1 -- 172.21.15.97:0/30464658 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 345512688 0 0) 0x7f87c00d3d00 con 0x7f87c015eae0 2024-03-29T04:07:19.790 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.809+0000 7f87be7fc640 1 -- 172.21.15.97:0/30464658 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87ac003520 con 0x7f87c015eae0 2024-03-29T04:07:19.790 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.809+0000 7f87be7fc640 1 -- 172.21.15.97:0/30464658 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 455336445 0 0) 0x7f87ac003520 con 0x7f87c015eae0 2024-03-29T04:07:19.790 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.809+0000 7f87bf7fe640 1 -- 172.21.15.97:0/30464658 >> v1:172.21.15.154:6789/0 conn(0x7f87c00d1630 legacy=0x7f87c00b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:19.790 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.809+0000 7f87bf7fe640 1 -- 172.21.15.97:0/30464658 >> v1:172.21.15.97:6789/0 conn(0x7f87c015eae0 legacy=0x7f87c00d35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:19.790 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.809+0000 7f87bf7fe640 1 -- 172.21.15.97:0/30464658 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c00d2b10 con 0x7f87c00b8260 2024-03-29T04:07:19.791 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.810+0000 7f87bf7fe640 1 --2- 172.21.15.97:0/30464658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00b8260 0x7f87c00d1a90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009d90 tx=0x7f87b802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4e459c643f59faf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:19.791 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.810+0000 7f87be7fc640 1 -- 172.21.15.97:0/30464658 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8039070 con 0x7f87c00b8260 2024-03-29T04:07:19.792 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.810+0000 7f87be7fc640 1 -- 172.21.15.97:0/30464658 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87b802c940 con 0x7f87c00b8260 2024-03-29T04:07:19.792 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.810+0000 7f87be7fc640 1 -- 172.21.15.97:0/30464658 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0001670 con 0x7f87c00b8260 2024-03-29T04:07:19.792 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.811+0000 7f87c5760640 1 -- 172.21.15.97:0/30464658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00b8260 msgr2=0x7f87c00d1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:19.792 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.811+0000 7f87c5760640 1 --2- 172.21.15.97:0/30464658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00b8260 0x7f87c00d1a90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009d90 tx=0x7f87b802be10 comp rx=0 tx=0).stop 2024-03-29T04:07:19.792 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.811+0000 7f87c5760640 1 -- 172.21.15.97:0/30464658 shutdown_connections 2024-03-29T04:07:19.792 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.811+0000 7f87c5760640 1 --2- 172.21.15.97:0/30464658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00b8260 0x7f87c00d1a90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:19.792 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.811+0000 7f87c5760640 1 -- 172.21.15.97:0/30464658 >> 172.21.15.97:0/30464658 conn(0x7f87c006cd00 msgr2=0x7f87c006d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:19.792 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.811+0000 7f87c5760640 1 -- 172.21.15.97:0/30464658 shutdown_connections 2024-03-29T04:07:19.792 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.811+0000 7f87c5760640 1 -- 172.21.15.97:0/30464658 wait complete. 2024-03-29T04:07:19.792 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.812+0000 7f87c5760640 1 Processor -- start 2024-03-29T04:07:19.792 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.812+0000 7f87c5760640 1 -- start start 2024-03-29T04:07:19.795 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87c5760640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c00b8260 0x7f87c01f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:19.795 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87c5760640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00d1630 0x7f87c01e9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:19.795 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87c5760640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87c015eae0 0x7f87c00ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:19.795 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87c5760640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f87c015eec0 con 0x7f87c00b8260 2024-03-29T04:07:19.795 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87c5760640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f87c00d30d0 con 0x7f87c00d1630 2024-03-29T04:07:19.795 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87c5760640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f87c015f040 con 0x7f87c015eae0 2024-03-29T04:07:19.795 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87beffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c00b8260 0x7f87c01f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:19.795 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87beffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c00b8260 0x7f87c01f5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49704/0 (socket says 172.21.15.97:49704) 2024-03-29T04:07:19.795 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87beffd640 1 -- 172.21.15.97:0/4081898939 learned_addr learned my addr 172.21.15.97:0/4081898939 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:19.795 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87bf7fe640 1 --2- 172.21.15.97:0/4081898939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87c015eae0 0x7f87c00ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:19.795 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87b7fff640 1 --2- 172.21.15.97:0/4081898939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00d1630 0x7f87c01e9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87beffd640 1 -- 172.21.15.97:0/4081898939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87c015eae0 msgr2=0x7f87c00ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87beffd640 1 --2- 172.21.15.97:0/4081898939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87c015eae0 0x7f87c00ac4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87beffd640 1 -- 172.21.15.97:0/4081898939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00d1630 msgr2=0x7f87c01e9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87beffd640 1 --2- 172.21.15.97:0/4081898939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00d1630 0x7f87c01e9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87beffd640 1 -- 172.21.15.97:0/4081898939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c00caac0 con 0x7f87c00b8260 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87beffd640 1 --2- 172.21.15.97:0/4081898939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c00b8260 0x7f87c01f5960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f87b000c520 tx=0x7f87b000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87b77fe640 1 -- 172.21.15.97:0/4081898939 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0017070 con 0x7f87c00b8260 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87b77fe640 1 -- 172.21.15.97:0/4081898939 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87b0005d00 con 0x7f87c00b8260 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87c5760640 1 -- 172.21.15.97:0/4081898939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c00ab400 con 0x7f87c00b8260 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87b77fe640 1 -- 172.21.15.97:0/4081898939 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b00072b0 con 0x7f87c00b8260 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87c5760640 1 -- 172.21.15.97:0/4081898939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c00ab9c0 con 0x7f87c00b8260 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87b77fe640 1 -- 172.21.15.97:0/4081898939 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f87b001a070 con 0x7f87c00b8260 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.813+0000 7f87b77fe640 1 -- 172.21.15.97:0/4081898939 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f87b0012ad0 con 0x7f87c00b8260 2024-03-29T04:07:19.796 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.814+0000 7f87c5760640 1 -- 172.21.15.97:0/4081898939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c004f9d0 con 0x7f87c00b8260 2024-03-29T04:07:19.797 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.816+0000 7f87b77fe640 1 -- 172.21.15.97:0/4081898939 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f87b0021020 con 0x7f87c00b8260 2024-03-29T04:07:19.857 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.877+0000 7f87c5760640 1 -- 172.21.15.97:0/4081898939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87c00d2620 con 0x7f87c00b8260 2024-03-29T04:07:19.858 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.878+0000 7f87b77fe640 1 -- 172.21.15.97:0/4081898939 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f87b0007660 con 0x7f87c00b8260 2024-03-29T04:07:19.858 INFO:teuthology.misc.health.smithi097.stdout: 2024-03-29T04:07:19.858 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":10,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:18.654511+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:19.859 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.879+0000 7f87c5760640 1 -- 172.21.15.97:0/4081898939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c00b8260 msgr2=0x7f87c01f5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:19.859 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.879+0000 7f87c5760640 1 --2- 172.21.15.97:0/4081898939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c00b8260 0x7f87c01f5960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f87b000c520 tx=0x7f87b000c9e0 comp rx=0 tx=0).stop 2024-03-29T04:07:19.860 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.879+0000 7f87c5760640 1 -- 172.21.15.97:0/4081898939 shutdown_connections 2024-03-29T04:07:19.860 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.879+0000 7f87c5760640 1 --2- 172.21.15.97:0/4081898939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87c015eae0 0x7f87c00ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:19.860 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.879+0000 7f87c5760640 1 --2- 172.21.15.97:0/4081898939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c00d1630 0x7f87c01e9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:19.860 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.879+0000 7f87c5760640 1 --2- 172.21.15.97:0/4081898939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87c00b8260 0x7f87c01f5960 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:19.860 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.879+0000 7f87c5760640 1 -- 172.21.15.97:0/4081898939 >> 172.21.15.97:0/4081898939 conn(0x7f87c006cd00 msgr2=0x7f87c00ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:19.860 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.879+0000 7f87c5760640 1 -- 172.21.15.97:0/4081898939 shutdown_connections 2024-03-29T04:07:19.860 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:19.879+0000 7f87c5760640 1 -- 172.21.15.97:0/4081898939 wait complete. 2024-03-29T04:07:19.868 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-29T04:07:20.158 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:20.178+0000 7f083402b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-29T04:07:20.228 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:20.247+0000 7f659be1a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-29T04:07:20.439 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:20.460+0000 7f083402b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-29T04:07:20.491 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:20.511+0000 7f083402b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-29T04:07:20.509 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:20.528+0000 7f659be1a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-29T04:07:20.560 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:20.580+0000 7f659be1a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-29T04:07:20.607 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:20.627+0000 7f083402b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-29T04:07:20.683 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-03-29T04:07:20.702+0000 7f659be1a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-29T04:07:21.622 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:21.642+0000 7f07d4426640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-29T04:07:21.622 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-03-29T04:07:21.642+0000 7f07d4426640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-29T04:07:22.738 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-03-29T04:07:22.758+0000 7f29cec21640 -1 osd.1 0 log_to_monitors true 2024-03-29T04:07:22.771 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-03-29T04:07:22.791+0000 7f5c0661e640 -1 osd.6 0 log_to_monitors true 2024-03-29T04:07:22.788 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-03-29T04:07:22.808+0000 7f31ef448640 -1 osd.2 0 log_to_monitors true 2024-03-29T04:07:22.789 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-03-29T04:07:22.810+0000 7ff1e981b640 -1 osd.0 0 log_to_monitors true 2024-03-29T04:07:22.818 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-03-29T04:07:22.838+0000 7fc6e0f2a640 -1 osd.3 0 log_to_monitors true 2024-03-29T04:07:22.842 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-03-29T04:07:22.861+0000 7fc1a7814640 -1 osd.7 0 log_to_monitors true 2024-03-29T04:07:22.846 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-03-29T04:07:22.866+0000 7f7abe0f7640 -1 osd.4 0 log_to_monitors true 2024-03-29T04:07:22.848 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-03-29T04:07:22.867+0000 7fbf2141f640 -1 osd.5 0 log_to_monitors true 2024-03-29T04:07:26.597 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-03-29T04:07:26.617+0000 7fbf1d580640 -1 osd.5 0 waiting for initial osdmap 2024-03-29T04:07:26.610 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-03-29T04:07:26.631+0000 7fbf14b8c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:07:26.633 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-03-29T04:07:26.654+0000 7f29ca56c640 -1 osd.1 0 waiting for initial osdmap 2024-03-29T04:07:26.641 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-03-29T04:07:26.661+0000 7f29c238c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:07:26.676 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:26.688 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-03-29T04:07:26.709+0000 7ff1e51ec640 -1 osd.0 0 waiting for initial osdmap 2024-03-29T04:07:26.703 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-03-29T04:07:26.724+0000 7ff1dd00c640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:07:26.711 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-03-29T04:07:26.731+0000 7fc6dcec0640 -1 osd.3 0 waiting for initial osdmap 2024-03-29T04:07:26.720 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-03-29T04:07:26.741+0000 7fc6d4ce0640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:07:26.726 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-03-29T04:07:26.747+0000 7f7aba08d640 -1 osd.4 0 waiting for initial osdmap 2024-03-29T04:07:26.738 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-03-29T04:07:26.759+0000 7f7ab1ead640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:07:26.750 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-03-29T04:07:26.770+0000 7f31eb3de640 -1 osd.2 0 waiting for initial osdmap 2024-03-29T04:07:26.762 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-03-29T04:07:26.783+0000 7f31e31fe640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:07:26.764 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-03-29T04:07:26.784+0000 7fc1a31ec640 -1 osd.7 0 waiting for initial osdmap 2024-03-29T04:07:26.768 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-03-29T04:07:26.789+0000 7f5c01f61640 -1 osd.6 0 waiting for initial osdmap 2024-03-29T04:07:26.775 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-03-29T04:07:26.796+0000 7fc19b00c640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:07:26.776 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.796+0000 7f24cdcd9640 1 Processor -- start 2024-03-29T04:07:26.776 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.796+0000 7f24cdcd9640 1 -- start start 2024-03-29T04:07:26.778 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24cdcd9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24c815e720 0x7f24c81623d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:26.778 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24cdcd9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f24c806d7e0 0x7f24c80b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:26.778 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24cdcd9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24c80d1980 con 0x7f24c8162910 2024-03-29T04:07:26.778 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24cdcd9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f24c8162d50 con 0x7f24c806d7e0 2024-03-29T04:07:26.778 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24cdcd9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f24c80d08e0 con 0x7f24c815e720 2024-03-29T04:07:26.778 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24c7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f24c806d7e0 0x7f24c80b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:26.778 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24c77fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24c815e720 0x7f24c81623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:26.778 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24c6ffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f24c8162910 0x7f24c806d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:50684/0 (socket says 172.21.15.97:50684) 2024-03-29T04:07:26.778 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24c7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f24c806d7e0 0x7f24c80b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:58366/0 (socket says 172.21.15.97:58366) 2024-03-29T04:07:26.779 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24c77fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24c815e720 0x7f24c81623d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:45646/0 (socket says 172.21.15.97:45646) 2024-03-29T04:07:26.779 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24c6ffd640 1 -- 172.21.15.97:0/2105798049 learned_addr learned my addr 172.21.15.97:0/2105798049 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:26.779 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24c7fff640 1 -- 172.21.15.97:0/2105798049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24c815e720 msgr2=0x7f24c81623d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:26.779 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24c7fff640 1 --2- 172.21.15.97:0/2105798049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24c815e720 0x7f24c81623d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:26.779 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24c7fff640 1 -- 172.21.15.97:0/2105798049 >> v1:172.21.15.154:6789/0 conn(0x7f24c8162910 legacy=0x7f24c806d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:26.779 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24c7fff640 1 -- 172.21.15.97:0/2105798049 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c80d0bc0 con 0x7f24c806d7e0 2024-03-29T04:07:26.779 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24c67fc640 1 -- 172.21.15.97:0/2105798049 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252503791 0 0) 0x7f24c80d1980 con 0x7f24c8162910 2024-03-29T04:07:26.779 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.797+0000 7f24c77fe640 1 --2- 172.21.15.97:0/2105798049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24c815e720 0x7f24c81623d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:26.779 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.798+0000 7f24c7fff640 1 --2- 172.21.15.97:0/2105798049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c806d7e0 0x7f24c80b31a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009d90 tx=0x7f24c002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=552c6e4fdb4c3117 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:26.779 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.798+0000 7f24c67fc640 1 -- 172.21.15.97:0/2105798049 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0039070 con 0x7f24c806d7e0 2024-03-29T04:07:26.779 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.798+0000 7f24c67fc640 1 -- 172.21.15.97:0/2105798049 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c002cc50 con 0x7f24c806d7e0 2024-03-29T04:07:26.779 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.798+0000 7f24c67fc640 1 -- 172.21.15.97:0/2105798049 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0034440 con 0x7f24c806d7e0 2024-03-29T04:07:26.782 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.799+0000 7f24cdcd9640 1 -- 172.21.15.97:0/2105798049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c806d7e0 msgr2=0x7f24c80b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:26.782 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.799+0000 7f24cdcd9640 1 --2- 172.21.15.97:0/2105798049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c806d7e0 0x7f24c80b31a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009d90 tx=0x7f24c002bea0 comp rx=0 tx=0).stop 2024-03-29T04:07:26.782 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.799+0000 7f24cdcd9640 1 -- 172.21.15.97:0/2105798049 shutdown_connections 2024-03-29T04:07:26.782 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.799+0000 7f24cdcd9640 1 --2- 172.21.15.97:0/2105798049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c806d7e0 0x7f24c80b31a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:26.782 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.799+0000 7f24cdcd9640 1 --2- 172.21.15.97:0/2105798049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24c815e720 0x7f24c81623d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:26.782 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.799+0000 7f24cdcd9640 1 -- 172.21.15.97:0/2105798049 >> 172.21.15.97:0/2105798049 conn(0x7f24c80d2460 msgr2=0x7f24c80d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:26.782 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.799+0000 7f24cdcd9640 1 -- 172.21.15.97:0/2105798049 shutdown_connections 2024-03-29T04:07:26.782 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.799+0000 7f24cdcd9640 1 -- 172.21.15.97:0/2105798049 wait complete. 2024-03-29T04:07:26.782 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.800+0000 7f24cdcd9640 1 Processor -- start 2024-03-29T04:07:26.783 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.800+0000 7f24cdcd9640 1 -- start start 2024-03-29T04:07:26.783 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24cdcd9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24c806d7e0 0x7f24c81f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:26.783 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24cdcd9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c815e720 0x7f24c80b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:26.783 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24cdcd9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24c8162910 0x7f24c81f1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:26.783 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24cdcd9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80d1460 con 0x7f24c806d7e0 2024-03-29T04:07:26.783 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24cdcd9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80d1180 con 0x7f24c815e720 2024-03-29T04:07:26.783 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24cdcd9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f24c80d05f0 con 0x7f24c8162910 2024-03-29T04:07:26.783 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24c806d7e0 0x7f24c81f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:26.783 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24c806d7e0 0x7f24c81f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37002/0 (socket says 172.21.15.97:37002) 2024-03-29T04:07:26.783 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24c77fe640 1 -- 172.21.15.97:0/3680449515 learned_addr learned my addr 172.21.15.97:0/3680449515 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:26.783 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24c7fff640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24c8162910 0x7f24c81f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:26.783 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24c6ffd640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c815e720 0x7f24c80b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:26.783 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24c77fe640 1 -- 172.21.15.97:0/3680449515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24c8162910 msgr2=0x7f24c81f1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:26.784 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24c77fe640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24c8162910 0x7f24c81f1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:26.784 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24c77fe640 1 -- 172.21.15.97:0/3680449515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c815e720 msgr2=0x7f24c80b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:26.784 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24c77fe640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c815e720 0x7f24c80b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:26.784 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24c77fe640 1 -- 172.21.15.97:0/3680449515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c80b30d0 con 0x7f24c806d7e0 2024-03-29T04:07:26.784 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24c77fe640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24c806d7e0 0x7f24c81f4e40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f24b800ec10 tx=0x7f24b800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:26.784 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.801+0000 7f24affff640 1 -- 172.21.15.97:0/3680449515 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8017070 con 0x7f24c806d7e0 2024-03-29T04:07:26.784 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.802+0000 7f24affff640 1 -- 172.21.15.97:0/3680449515 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b800cc30 con 0x7f24c806d7e0 2024-03-29T04:07:26.784 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.802+0000 7f24cdcd9640 1 -- 172.21.15.97:0/3680449515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c80b33a0 con 0x7f24c806d7e0 2024-03-29T04:07:26.784 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.802+0000 7f24affff640 1 -- 172.21.15.97:0/3680449515 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8013470 con 0x7f24c806d7e0 2024-03-29T04:07:26.784 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.802+0000 7f24cdcd9640 1 -- 172.21.15.97:0/3680449515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c80ca540 con 0x7f24c806d7e0 2024-03-29T04:07:26.785 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.803+0000 7f24cdcd9640 1 -- 172.21.15.97:0/3680449515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c804f9d0 con 0x7f24c806d7e0 2024-03-29T04:07:26.785 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.803+0000 7f24affff640 1 -- 172.21.15.97:0/3680449515 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 90800+0+0 (secure 0 0 0) 0x7f24b801a070 con 0x7f24c806d7e0 2024-03-29T04:07:26.785 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.803+0000 7f24affff640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f24b406c4a0 0x7f24b406e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:26.785 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.803+0000 7f24c6ffd640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f24b406c4a0 0x7f24b406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:26.785 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.803+0000 7f24affff640 1 -- 172.21.15.97:0/3680449515 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f24b808a6c0 con 0x7f24c806d7e0 2024-03-29T04:07:26.785 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.804+0000 7f24c6ffd640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f24b406c4a0 0x7f24b406e960 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f24bc007fd0 tx=0x7f24bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:26.786 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-03-29T04:07:26.807+0000 7f5bf9d92640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:07:26.787 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.808+0000 7f24affff640 1 -- 172.21.15.97:0/3680449515 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f24b8059be0 con 0x7f24c806d7e0 2024-03-29T04:07:26.963 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.984+0000 7f24cdcd9640 1 -- 172.21.15.97:0/3680449515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24c80ca950 con 0x7f24c806d7e0 2024-03-29T04:07:26.964 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.984+0000 7f24affff640 1 -- 172.21.15.97:0/3680449515 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f24b80594e0 con 0x7f24c806d7e0 2024-03-29T04:07:26.964 INFO:teuthology.misc.health.smithi097.stdout: 2024-03-29T04:07:26.964 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":12,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:24.645172+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:26.967 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.986+0000 7f24cdcd9640 1 -- 172.21.15.97:0/3680449515 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f24b406c4a0 msgr2=0x7f24b406e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:26.967 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.986+0000 7f24cdcd9640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f24b406c4a0 0x7f24b406e960 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f24bc007fd0 tx=0x7f24bc009040 comp rx=0 tx=0).stop 2024-03-29T04:07:26.967 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.986+0000 7f24cdcd9640 1 -- 172.21.15.97:0/3680449515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24c806d7e0 msgr2=0x7f24c81f4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:26.967 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.986+0000 7f24cdcd9640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24c806d7e0 0x7f24c81f4e40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f24b800ec10 tx=0x7f24b800c520 comp rx=0 tx=0).stop 2024-03-29T04:07:26.967 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.986+0000 7f24cdcd9640 1 -- 172.21.15.97:0/3680449515 shutdown_connections 2024-03-29T04:07:26.967 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.986+0000 7f24cdcd9640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f24b406c4a0 0x7f24b406e960 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:26.967 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.986+0000 7f24cdcd9640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f24c8162910 0x7f24c81f1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:26.967 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.986+0000 7f24cdcd9640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c815e720 0x7f24c80b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:26.967 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.986+0000 7f24cdcd9640 1 --2- 172.21.15.97:0/3680449515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24c806d7e0 0x7f24c81f4e40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:26.967 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.986+0000 7f24cdcd9640 1 -- 172.21.15.97:0/3680449515 >> 172.21.15.97:0/3680449515 conn(0x7f24c80d2460 msgr2=0x7f24c8067410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:26.967 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.986+0000 7f24cdcd9640 1 -- 172.21.15.97:0/3680449515 shutdown_connections 2024-03-29T04:07:26.967 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:26.986+0000 7f24cdcd9640 1 -- 172.21.15.97:0/3680449515 wait complete. 2024-03-29T04:07:26.976 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-29T04:07:33.783 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:33.883 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.903+0000 7fe44b10e640 1 Processor -- start 2024-03-29T04:07:33.884 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.904+0000 7fe44b10e640 1 -- start start 2024-03-29T04:07:33.885 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.904+0000 7fe44b10e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe4440c66c0 0x7fe444167500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:33.885 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.904+0000 7fe44b10e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4440d0030 con 0x7fe4440d0dd0 2024-03-29T04:07:33.885 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.904+0000 7fe44b10e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4440ce1d0 con 0x7fe44415c530 2024-03-29T04:07:33.885 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.904+0000 7fe44b10e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe4440d0940 con 0x7fe4440c66c0 2024-03-29T04:07:33.885 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.904+0000 7fe4496ab640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe4440c66c0 0x7fe444167500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:33.885 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.904+0000 7fe443fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe4440d0dd0 0x7fe4440c5fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52606/0 (socket says 172.21.15.97:52606) 2024-03-29T04:07:33.885 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.904+0000 7fe4496ab640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe4440c66c0 0x7fe444167500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60098/0 (socket says 172.21.15.97:60098) 2024-03-29T04:07:33.885 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.904+0000 7fe443fff640 1 -- 172.21.15.97:0/2424412545 learned_addr learned my addr 172.21.15.97:0/2424412545 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:33.885 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.905+0000 7fe4496ab640 1 -- 172.21.15.97:0/2424412545 >> v1:172.21.15.154:6789/0 conn(0x7fe44415c530 legacy=0x7fe4440cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:33.885 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.905+0000 7fe4496ab640 1 -- 172.21.15.97:0/2424412545 >> v1:172.21.15.97:6789/0 conn(0x7fe4440d0dd0 legacy=0x7fe4440c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:33.886 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.905+0000 7fe4496ab640 1 -- 172.21.15.97:0/2424412545 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4440d02f0 con 0x7fe4440c66c0 2024-03-29T04:07:33.886 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.905+0000 7fe4437fe640 1 -- 172.21.15.97:0/2424412545 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2173479832 0 0) 0x7fe4440d0030 con 0x7fe4440d0dd0 2024-03-29T04:07:33.887 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.905+0000 7fe4496ab640 1 --2- 172.21.15.97:0/2424412545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440c66c0 0x7fe444167500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0097a0 tx=0x7fe43c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4c8d59a47856e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:33.887 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.906+0000 7fe4437fe640 1 -- 172.21.15.97:0/2424412545 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c034070 con 0x7fe4440c66c0 2024-03-29T04:07:33.887 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.906+0000 7fe4437fe640 1 -- 172.21.15.97:0/2424412545 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe43c030610 con 0x7fe4440c66c0 2024-03-29T04:07:33.887 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.906+0000 7fe4437fe640 1 -- 172.21.15.97:0/2424412545 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c0309e0 con 0x7fe4440c66c0 2024-03-29T04:07:33.887 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.906+0000 7fe44b10e640 1 -- 172.21.15.97:0/2424412545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440c66c0 msgr2=0x7fe444167500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:33.888 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.906+0000 7fe44b10e640 1 --2- 172.21.15.97:0/2424412545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440c66c0 0x7fe444167500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0097a0 tx=0x7fe43c02b960 comp rx=0 tx=0).stop 2024-03-29T04:07:33.888 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.906+0000 7fe44b10e640 1 -- 172.21.15.97:0/2424412545 shutdown_connections 2024-03-29T04:07:33.888 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.906+0000 7fe44b10e640 1 --2- 172.21.15.97:0/2424412545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4440c66c0 0x7fe444167500 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:33.888 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.906+0000 7fe44b10e640 1 -- 172.21.15.97:0/2424412545 >> 172.21.15.97:0/2424412545 conn(0x7fe4440d2460 msgr2=0x7fe4440d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:33.888 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.906+0000 7fe44b10e640 1 -- 172.21.15.97:0/2424412545 shutdown_connections 2024-03-29T04:07:33.888 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.906+0000 7fe44b10e640 1 -- 172.21.15.97:0/2424412545 wait complete. 2024-03-29T04:07:33.888 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.907+0000 7fe44b10e640 1 Processor -- start 2024-03-29T04:07:33.888 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.907+0000 7fe44b10e640 1 -- start start 2024-03-29T04:07:33.891 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.908+0000 7fe44b10e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4440c66c0 0x7fe4441599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:33.891 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.908+0000 7fe44b10e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440d0dd0 0x7fe444070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:33.891 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.908+0000 7fe44b10e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe44415c530 0x7fe444071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:33.891 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.908+0000 7fe44b10e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe44415c910 con 0x7fe44415c530 2024-03-29T04:07:33.891 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.908+0000 7fe44b10e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440c6ae0 con 0x7fe4440d0dd0 2024-03-29T04:07:33.891 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.908+0000 7fe44b10e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe44415ca90 con 0x7fe4440c66c0 2024-03-29T04:07:33.891 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.908+0000 7fe4496ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe44415c530 0x7fe444071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:33.891 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.908+0000 7fe4496ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe44415c530 0x7fe444071430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60112/0 (socket says 172.21.15.97:60112) 2024-03-29T04:07:33.891 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.908+0000 7fe4496ab640 1 -- 172.21.15.97:0/3478775300 learned_addr learned my addr 172.21.15.97:0/3478775300 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:33.891 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.909+0000 7fe443fff640 1 --2- 172.21.15.97:0/3478775300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440d0dd0 0x7fe444070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:33.891 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.909+0000 7fe4496ab640 1 -- 172.21.15.97:0/3478775300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4440c66c0 msgr2=0x7fe4441599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:33.891 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.909+0000 7fe4496ab640 1 --2- 172.21.15.97:0/3478775300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4440c66c0 0x7fe4441599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:33.891 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.909+0000 7fe4496ab640 1 -- 172.21.15.97:0/3478775300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440d0dd0 msgr2=0x7fe444070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:33.892 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.909+0000 7fe4496ab640 1 --2- 172.21.15.97:0/3478775300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440d0dd0 0x7fe444070ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:33.892 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.909+0000 7fe4496ab640 1 -- 172.21.15.97:0/3478775300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4440cb660 con 0x7fe44415c530 2024-03-29T04:07:33.892 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.909+0000 7fe4496ab640 1 --2- 172.21.15.97:0/3478775300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe44415c530 0x7fe444071430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0098c0 tx=0x7fe43c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:33.892 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.909+0000 7fe4417fa640 1 -- 172.21.15.97:0/3478775300 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c034040 con 0x7fe44415c530 2024-03-29T04:07:33.892 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.909+0000 7fe4417fa640 1 -- 172.21.15.97:0/3478775300 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe43c039990 con 0x7fe44415c530 2024-03-29T04:07:33.892 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.909+0000 7fe4417fa640 1 -- 172.21.15.97:0/3478775300 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c039d80 con 0x7fe44415c530 2024-03-29T04:07:33.892 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.909+0000 7fe44b10e640 1 -- 172.21.15.97:0/3478775300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4440cb940 con 0x7fe44415c530 2024-03-29T04:07:33.892 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.909+0000 7fe44b10e640 1 -- 172.21.15.97:0/3478775300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4440cbf00 con 0x7fe44415c530 2024-03-29T04:07:33.892 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.910+0000 7fe44b10e640 1 -- 172.21.15.97:0/3478775300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe444068750 con 0x7fe44415c530 2024-03-29T04:07:33.892 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.911+0000 7fe4417fa640 1 -- 172.21.15.97:0/3478775300 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe43c037070 con 0x7fe44415c530 2024-03-29T04:07:33.892 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.911+0000 7fe4417fa640 1 --2- 172.21.15.97:0/3478775300 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe42406c530 0x7fe42406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:33.892 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.911+0000 7fe4417fa640 1 -- 172.21.15.97:0/3478775300 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe43c0ad480 con 0x7fe44415c530 2024-03-29T04:07:33.892 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.911+0000 7fe448eaa640 1 --2- 172.21.15.97:0/3478775300 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe42406c530 0x7fe42406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:33.893 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.911+0000 7fe448eaa640 1 --2- 172.21.15.97:0/3478775300 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe42406c530 0x7fe42406e9f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe42c000f60 tx=0x7fe42c0079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:33.895 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:33.916+0000 7fe4417fa640 1 -- 172.21.15.97:0/3478775300 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe444068750 con 0x7fe44415c530 2024-03-29T04:07:34.071 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.090+0000 7fe44b10e640 1 -- 172.21.15.97:0/3478775300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4440d3050 con 0x7fe44415c530 2024-03-29T04:07:34.071 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.091+0000 7fe4417fa640 1 -- 172.21.15.97:0/3478775300 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7fe43c07bdf0 con 0x7fe44415c530 2024-03-29T04:07:34.072 INFO:teuthology.misc.health.smithi097.stdout: 2024-03-29T04:07:34.072 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":16,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:30.780150+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:34.073 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.093+0000 7fe44b10e640 1 -- 172.21.15.97:0/3478775300 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe42406c530 msgr2=0x7fe42406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.073 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.093+0000 7fe44b10e640 1 --2- 172.21.15.97:0/3478775300 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe42406c530 0x7fe42406e9f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe42c000f60 tx=0x7fe42c0079b0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.073 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.093+0000 7fe44b10e640 1 -- 172.21.15.97:0/3478775300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe44415c530 msgr2=0x7fe444071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.073 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.093+0000 7fe44b10e640 1 --2- 172.21.15.97:0/3478775300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe44415c530 0x7fe444071430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0098c0 tx=0x7fe43c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.073 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.093+0000 7fe44b10e640 1 -- 172.21.15.97:0/3478775300 shutdown_connections 2024-03-29T04:07:34.073 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.093+0000 7fe44b10e640 1 --2- 172.21.15.97:0/3478775300 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe42406c530 0x7fe42406e9f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.074 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.093+0000 7fe44b10e640 1 --2- 172.21.15.97:0/3478775300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe44415c530 0x7fe444071430 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.074 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.093+0000 7fe44b10e640 1 --2- 172.21.15.97:0/3478775300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440d0dd0 0x7fe444070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.074 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.093+0000 7fe44b10e640 1 --2- 172.21.15.97:0/3478775300 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4440c66c0 0x7fe4441599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.074 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.093+0000 7fe44b10e640 1 -- 172.21.15.97:0/3478775300 >> 172.21.15.97:0/3478775300 conn(0x7fe4440d2460 msgr2=0x7fe4440c5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:34.075 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.093+0000 7fe44b10e640 1 -- 172.21.15.97:0/3478775300 shutdown_connections 2024-03-29T04:07:34.075 INFO:teuthology.misc.health.smithi097.stderr:2024-03-29T04:07:34.093+0000 7fe44b10e640 1 -- 172.21.15.97:0/3478775300 wait complete. 2024-03-29T04:07:34.083 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-29T04:07:34.083 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-29T04:07:34.083 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T04:07:34.084 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T04:07:34.086 INFO:tasks.ceph.mds.a:Started 2024-03-29T04:07:34.086 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T04:07:34.086 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T04:07:34.088 INFO:tasks.ceph.mds.c:Started 2024-03-29T04:07:34.089 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T04:07:34.089 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T04:07:34.091 INFO:tasks.ceph.mds.b:Started 2024-03-29T04:07:34.092 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T04:07:34.092 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T04:07:34.094 INFO:tasks.ceph.mds.d:Started 2024-03-29T04:07:34.094 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-29T04:07:34.095 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:34.162 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-03-29T04:07:34.162 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-03-29T04:07:34.176 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2024-03-29T04:07:34.177 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2024-03-29T04:07:34.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.219+0000 7fd682a70640 1 Processor -- start 2024-03-29T04:07:34.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.219+0000 7fd682a70640 1 -- start start 2024-03-29T04:07:34.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.220+0000 7fd682a70640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd67c15f9d0 0x7fd67c162d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.220+0000 7fd682a70640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd67c0d4e20 0x7fd67c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.220+0000 7fd682a70640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd67c0cf390 con 0x7fd67c0d1150 2024-03-29T04:07:34.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.220+0000 7fd682a70640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fd67c0d06d0 con 0x7fd67c15f9d0 2024-03-29T04:07:34.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.220+0000 7fd682a70640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fd67c0d0240 con 0x7fd67c0d4e20 2024-03-29T04:07:34.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.220+0000 7fd67bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd67c15f9d0 0x7fd67c162d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.220+0000 7fd67bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd67c15f9d0 0x7fd67c162d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60154/0 (socket says 172.21.15.97:60154) 2024-03-29T04:07:34.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.220+0000 7fd67bfff640 1 -- 172.21.15.97:0/2947079377 learned_addr learned my addr 172.21.15.97:0/2947079377 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:34.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.220+0000 7fd67bfff640 1 -- 172.21.15.97:0/2947079377 >> v2:172.21.15.154:3300/0 conn(0x7fd67c0d4e20 msgr2=0x7fd67c0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:34.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.220+0000 7fd67bfff640 1 --2- 172.21.15.97:0/2947079377 >> v2:172.21.15.154:3300/0 conn(0x7fd67c0d4e20 0x7fd67c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.220+0000 7fd67bfff640 1 -- 172.21.15.97:0/2947079377 >> v1:172.21.15.154:6789/0 conn(0x7fd67c0d1150 legacy=0x7fd67c0d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:34.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.220+0000 7fd67bfff640 1 -- 172.21.15.97:0/2947079377 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c0cf630 con 0x7fd67c15f9d0 2024-03-29T04:07:34.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.221+0000 7fd67bfff640 1 --2- 172.21.15.97:0/2947079377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd67c15f9d0 0x7fd67c162d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd67c08ced0 tx=0x7fd67002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36e3dca677f9357 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.221+0000 7fd67b7fe640 1 -- 172.21.15.97:0/2947079377 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670035020 con 0x7fd67c15f9d0 2024-03-29T04:07:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.221+0000 7fd67b7fe640 1 -- 172.21.15.97:0/2947079377 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd670031870 con 0x7fd67c15f9d0 2024-03-29T04:07:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.221+0000 7fd67b7fe640 1 -- 172.21.15.97:0/2947079377 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670031c40 con 0x7fd67c15f9d0 2024-03-29T04:07:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.222+0000 7fd682a70640 1 -- 172.21.15.97:0/2947079377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd67c15f9d0 msgr2=0x7fd67c162d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.222+0000 7fd682a70640 1 --2- 172.21.15.97:0/2947079377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd67c15f9d0 0x7fd67c162d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd67c08ced0 tx=0x7fd67002b910 comp rx=0 tx=0).stop 2024-03-29T04:07:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.222+0000 7fd682a70640 1 -- 172.21.15.97:0/2947079377 shutdown_connections 2024-03-29T04:07:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.222+0000 7fd682a70640 1 --2- 172.21.15.97:0/2947079377 >> v2:172.21.15.154:3300/0 conn(0x7fd67c0d4e20 0x7fd67c0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.222+0000 7fd682a70640 1 --2- 172.21.15.97:0/2947079377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd67c15f9d0 0x7fd67c162d30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.222+0000 7fd682a70640 1 -- 172.21.15.97:0/2947079377 >> 172.21.15.97:0/2947079377 conn(0x7fd67c08bfc0 msgr2=0x7fd67c0d4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.222+0000 7fd682a70640 1 -- 172.21.15.97:0/2947079377 shutdown_connections 2024-03-29T04:07:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.222+0000 7fd682a70640 1 -- 172.21.15.97:0/2947079377 wait complete. 2024-03-29T04:07:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.223+0000 7fd682a70640 1 Processor -- start 2024-03-29T04:07:34.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.223+0000 7fd682a70640 1 -- start start 2024-03-29T04:07:34.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd682a70640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd67c0d1150 0x7fd67c158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd682a70640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd67c0d4e20 0x7fd67c159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd682a70640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67c15f9d0 0x7fd67c0b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd682a70640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c0cfed0 con 0x7fd67c0d4e20 2024-03-29T04:07:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd682a70640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c0cfbf0 con 0x7fd67c0d1150 2024-03-29T04:07:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd682a70640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd67c0d1530 con 0x7fd67c15f9d0 2024-03-29T04:07:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd67bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd67c0d4e20 0x7fd67c159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd67bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd67c0d4e20 0x7fd67c159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60164/0 (socket says 172.21.15.97:60164) 2024-03-29T04:07:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd67bfff640 1 -- 172.21.15.97:0/3465927564 learned_addr learned my addr 172.21.15.97:0/3465927564 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd68080c640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd67c0d1150 0x7fd67c158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd68100d640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67c15f9d0 0x7fd67c0b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd67bfff640 1 -- 172.21.15.97:0/3465927564 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67c15f9d0 msgr2=0x7fd67c0b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd67bfff640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67c15f9d0 0x7fd67c0b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd67bfff640 1 -- 172.21.15.97:0/3465927564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd67c0d1150 msgr2=0x7fd67c158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd67bfff640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd67c0d1150 0x7fd67c158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd67bfff640 1 -- 172.21.15.97:0/3465927564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c159c70 con 0x7fd67c0d4e20 2024-03-29T04:07:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd68100d640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67c15f9d0 0x7fd67c0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd68080c640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd67c0d1150 0x7fd67c158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd67bfff640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd67c0d4e20 0x7fd67c159500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6700090b0 tx=0x7fd6700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.224+0000 7fd6797fa640 1 -- 172.21.15.97:0/3465927564 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700305d0 con 0x7fd67c0d4e20 2024-03-29T04:07:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.225+0000 7fd6797fa640 1 -- 172.21.15.97:0/3465927564 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd67003ae50 con 0x7fd67c0d4e20 2024-03-29T04:07:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.225+0000 7fd682a70640 1 -- 172.21.15.97:0/3465927564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67c159f50 con 0x7fd67c0d4e20 2024-03-29T04:07:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.225+0000 7fd6797fa640 1 -- 172.21.15.97:0/3465927564 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700309b0 con 0x7fd67c0d4e20 2024-03-29T04:07:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.225+0000 7fd682a70640 1 -- 172.21.15.97:0/3465927564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67c158740 con 0x7fd67c0d4e20 2024-03-29T04:07:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.226+0000 7fd682a70640 1 -- 172.21.15.97:0/3465927564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67c068640 con 0x7fd67c0d4e20 2024-03-29T04:07:34.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.226+0000 7fd6797fa640 1 -- 172.21.15.97:0/3465927564 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd670038020 con 0x7fd67c0d4e20 2024-03-29T04:07:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.226+0000 7fd6797fa640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd66806c530 0x7fd66806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.226+0000 7fd6797fa640 1 -- 172.21.15.97:0/3465927564 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fd6700ad8e0 con 0x7fd67c0d4e20 2024-03-29T04:07:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.227+0000 7fd68080c640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd66806c530 0x7fd66806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.227+0000 7fd68080c640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd66806c530 0x7fd66806e9f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd66c001750 tx=0x7fd66c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:34.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.231+0000 7fd6797fa640 1 -- 172.21.15.97:0/3465927564 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd67003f070 con 0x7fd67c0d4e20 2024-03-29T04:07:34.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.407+0000 7fd682a70640 1 -- 172.21.15.97:0/3465927564 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd67c0d5570 con 0x7fd67c0d4e20 2024-03-29T04:07:34.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.408+0000 7fd6797fa640 1 -- 172.21.15.97:0/3465927564 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7fd67007c250 con 0x7fd67c0d4e20 2024-03-29T04:07:34.388 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:34.388 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:30.780150+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:34.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.410+0000 7fd682a70640 1 -- 172.21.15.97:0/3465927564 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd66806c530 msgr2=0x7fd66806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.410+0000 7fd682a70640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd66806c530 0x7fd66806e9f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd66c001750 tx=0x7fd66c009040 comp rx=0 tx=0).stop 2024-03-29T04:07:34.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.410+0000 7fd682a70640 1 -- 172.21.15.97:0/3465927564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd67c0d4e20 msgr2=0x7fd67c159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.410+0000 7fd682a70640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd67c0d4e20 0x7fd67c159500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6700090b0 tx=0x7fd6700046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.410+0000 7fd682a70640 1 -- 172.21.15.97:0/3465927564 shutdown_connections 2024-03-29T04:07:34.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.410+0000 7fd682a70640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd66806c530 0x7fd66806e9f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.411+0000 7fd682a70640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd67c15f9d0 0x7fd67c0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.411+0000 7fd682a70640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd67c0d4e20 0x7fd67c159500 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.411+0000 7fd682a70640 1 --2- 172.21.15.97:0/3465927564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd67c0d1150 0x7fd67c158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.411+0000 7fd682a70640 1 -- 172.21.15.97:0/3465927564 >> 172.21.15.97:0/3465927564 conn(0x7fd67c08bfc0 msgr2=0x7fd67c162800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:34.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.411+0000 7fd682a70640 1 -- 172.21.15.97:0/3465927564 shutdown_connections 2024-03-29T04:07:34.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.411+0000 7fd682a70640 1 -- 172.21.15.97:0/3465927564 wait complete. 2024-03-29T04:07:34.401 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:07:34.402 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:07:34.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.524+0000 7fa454de6640 1 Processor -- start 2024-03-29T04:07:34.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.524+0000 7fa454de6640 1 -- start start 2024-03-29T04:07:34.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.525+0000 7fa454de6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4500cef50 0x7fa45006b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.525+0000 7fa454de6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa45006ba50 0x7fa4501624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.525+0000 7fa454de6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4500b1c10 con 0x7fa45015c480 2024-03-29T04:07:34.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.525+0000 7fa454de6640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fa45006bea0 con 0x7fa45006ba50 2024-03-29T04:07:34.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.525+0000 7fa454de6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa4500b2d20 con 0x7fa4500cef50 2024-03-29T04:07:34.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44e59c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa45015c480 0x7fa4500b2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52624/0 (socket says 172.21.15.97:52624) 2024-03-29T04:07:34.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44e59c640 1 -- 172.21.15.97:0/3464298696 learned_addr learned my addr 172.21.15.97:0/3464298696 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:34.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44ed9d640 1 --2- 172.21.15.97:0/3464298696 >> v2:172.21.15.154:3300/0 conn(0x7fa45006ba50 0x7fa4501624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44dd9b640 1 --2- 172.21.15.97:0/3464298696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4500cef50 0x7fa45006b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 538017772 0 0) 0x7fa4500b1c10 con 0x7fa45015c480 2024-03-29T04:07:34.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa438003520 con 0x7fa45015c480 2024-03-29T04:07:34.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3400751725 0 0) 0x7fa438003520 con 0x7fa45015c480 2024-03-29T04:07:34.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4500cef50 msgr2=0x7fa45006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:34.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 --2- 172.21.15.97:0/3464298696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4500cef50 0x7fa45006b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 >> v2:172.21.15.154:3300/0 conn(0x7fa45006ba50 msgr2=0x7fa4501624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:34.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 --2- 172.21.15.97:0/3464298696 >> v2:172.21.15.154:3300/0 conn(0x7fa45006ba50 0x7fa4501624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4500b0e50 con 0x7fa45015c480 2024-03-29T04:07:34.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7fa448001670 con 0x7fa45015c480 2024-03-29T04:07:34.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 >> v1:172.21.15.97:6789/0 conn(0x7fa45015c480 legacy=0x7fa4500b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 --2- 172.21.15.97:0/3464298696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4380039b0 0x7fa438003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 --2- 172.21.15.97:0/3464298696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4500cef50 0x7fa438004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 --2- 172.21.15.97:0/3464298696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa438004d40 0x7fa4380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa45006bea0 con 0x7fa4380039b0 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4500b2d20 con 0x7fa438004d40 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa43800ba70 con 0x7fa4500cef50 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44e59c640 1 --2- 172.21.15.97:0/3464298696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4380039b0 0x7fa438003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44dd9b640 1 --2- 172.21.15.97:0/3464298696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4500cef50 0x7fa438004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44ed9d640 1 --2- 172.21.15.97:0/3464298696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa438004d40 0x7fa4380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44e59c640 1 -- 172.21.15.97:0/3464298696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4500cef50 msgr2=0x7fa438004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44e59c640 1 --2- 172.21.15.97:0/3464298696 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4500cef50 0x7fa438004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44e59c640 1 -- 172.21.15.97:0/3464298696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa438004d40 msgr2=0x7fa4380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44e59c640 1 --2- 172.21.15.97:0/3464298696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa438004d40 0x7fa4380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.526+0000 7fa44e59c640 1 -- 172.21.15.97:0/3464298696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa438003520 con 0x7fa4380039b0 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.527+0000 7fa44e59c640 1 --2- 172.21.15.97:0/3464298696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4380039b0 0x7fa438003dd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa448007fd0 tx=0x7fa438005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6243e03d235fb5ac server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.527+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448007d50 con 0x7fa4380039b0 2024-03-29T04:07:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.527+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa448030070 con 0x7fa4380039b0 2024-03-29T04:07:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.527+0000 7fa44d59a640 1 -- 172.21.15.97:0/3464298696 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44802cd40 con 0x7fa4380039b0 2024-03-29T04:07:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.528+0000 7fa454de6640 1 -- 172.21.15.97:0/3464298696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4380039b0 msgr2=0x7fa438003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.528+0000 7fa454de6640 1 --2- 172.21.15.97:0/3464298696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4380039b0 0x7fa438003dd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa448007fd0 tx=0x7fa438005830 comp rx=0 tx=0).stop 2024-03-29T04:07:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.528+0000 7fa44ed9d640 1 -- 172.21.15.97:0/3464298696 reap_dead start 2024-03-29T04:07:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.528+0000 7fa454de6640 1 -- 172.21.15.97:0/3464298696 shutdown_connections 2024-03-29T04:07:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.528+0000 7fa454de6640 1 -- 172.21.15.97:0/3464298696 >> 172.21.15.97:0/3464298696 conn(0x7fa4500b2ed0 msgr2=0x7fa4500b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.528+0000 7fa454de6640 1 -- 172.21.15.97:0/3464298696 shutdown_connections 2024-03-29T04:07:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.528+0000 7fa454de6640 1 -- 172.21.15.97:0/3464298696 wait complete. 2024-03-29T04:07:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.529+0000 7fa454de6640 1 Processor -- start 2024-03-29T04:07:34.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.529+0000 7fa454de6640 1 -- start start 2024-03-29T04:07:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa454de6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4380039b0 0x7fa4500b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa454de6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa450159e90 0x7fa4500b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa454de6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4501f86e0 0x7fa4501f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa454de6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4500b1410 con 0x7fa450159e90 2024-03-29T04:07:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa454de6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4500b1780 con 0x7fa4380039b0 2024-03-29T04:07:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa454de6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa45015c860 con 0x7fa4501f86e0 2024-03-29T04:07:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa44dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa450159e90 0x7fa4500b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa44dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa450159e90 0x7fa4500b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60178/0 (socket says 172.21.15.97:60178) 2024-03-29T04:07:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa44dd9b640 1 -- 172.21.15.97:0/2799566209 learned_addr learned my addr 172.21.15.97:0/2799566209 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa44ed9d640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4501f86e0 0x7fa4501f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa44e59c640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4380039b0 0x7fa4500b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa44dd9b640 1 -- 172.21.15.97:0/2799566209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4501f86e0 msgr2=0x7fa4501f8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa44dd9b640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4501f86e0 0x7fa4501f8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa44dd9b640 1 -- 172.21.15.97:0/2799566209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4380039b0 msgr2=0x7fa4500b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa44dd9b640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4380039b0 0x7fa4500b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa44dd9b640 1 -- 172.21.15.97:0/2799566209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4500b4560 con 0x7fa450159e90 2024-03-29T04:07:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa44e59c640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4380039b0 0x7fa4500b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa44dd9b640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa450159e90 0x7fa4500b6450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa444007f60 tx=0x7fa444007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.530+0000 7fa43effd640 1 -- 172.21.15.97:0/2799566209 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44401a020 con 0x7fa450159e90 2024-03-29T04:07:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.531+0000 7fa43effd640 1 -- 172.21.15.97:0/2799566209 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa444016470 con 0x7fa450159e90 2024-03-29T04:07:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.531+0000 7fa454de6640 1 -- 172.21.15.97:0/2799566209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4500b4840 con 0x7fa450159e90 2024-03-29T04:07:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.531+0000 7fa43effd640 1 -- 172.21.15.97:0/2799566209 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444016860 con 0x7fa450159e90 2024-03-29T04:07:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.531+0000 7fa454de6640 1 -- 172.21.15.97:0/2799566209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4501f81c0 con 0x7fa450159e90 2024-03-29T04:07:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.532+0000 7fa454de6640 1 -- 172.21.15.97:0/2799566209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa45009f530 con 0x7fa450159e90 2024-03-29T04:07:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.532+0000 7fa43effd640 1 -- 172.21.15.97:0/2799566209 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa444005d10 con 0x7fa450159e90 2024-03-29T04:07:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.532+0000 7fa43effd640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa43006c560 0x7fa43006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.532+0000 7fa43effd640 1 -- 172.21.15.97:0/2799566209 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa44408c230 con 0x7fa450159e90 2024-03-29T04:07:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.532+0000 7fa44e59c640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa43006c560 0x7fa43006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.533+0000 7fa44e59c640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa43006c560 0x7fa43006ea20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa448034000 tx=0x7fa448007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.536+0000 7fa43effd640 1 -- 172.21.15.97:0/2799566209 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa444024030 con 0x7fa450159e90 2024-03-29T04:07:34.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.710+0000 7fa454de6640 1 -- 172.21.15.97:0/2799566209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa45004f9d0 con 0x7fa450159e90 2024-03-29T04:07:34.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.710+0000 7fa43effd640 1 -- 172.21.15.97:0/2799566209 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fa44405aba0 con 0x7fa450159e90 2024-03-29T04:07:34.691 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:07:34.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.712+0000 7fa454de6640 1 -- 172.21.15.97:0/2799566209 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa43006c560 msgr2=0x7fa43006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.712+0000 7fa454de6640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa43006c560 0x7fa43006ea20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa448034000 tx=0x7fa448007000 comp rx=0 tx=0).stop 2024-03-29T04:07:34.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.712+0000 7fa454de6640 1 -- 172.21.15.97:0/2799566209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa450159e90 msgr2=0x7fa4500b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.712+0000 7fa454de6640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa450159e90 0x7fa4500b6450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa444007f60 tx=0x7fa444007f90 comp rx=0 tx=0).stop 2024-03-29T04:07:34.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.712+0000 7fa454de6640 1 -- 172.21.15.97:0/2799566209 shutdown_connections 2024-03-29T04:07:34.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.712+0000 7fa454de6640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa43006c560 0x7fa43006ea20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.712+0000 7fa454de6640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4501f86e0 0x7fa4501f8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.712+0000 7fa454de6640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa450159e90 0x7fa4500b6450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.712+0000 7fa454de6640 1 --2- 172.21.15.97:0/2799566209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4380039b0 0x7fa4500b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.712+0000 7fa454de6640 1 -- 172.21.15.97:0/2799566209 >> 172.21.15.97:0/2799566209 conn(0x7fa4500b2ed0 msgr2=0x7fa450157800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:34.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.712+0000 7fa454de6640 1 -- 172.21.15.97:0/2799566209 shutdown_connections 2024-03-29T04:07:34.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.712+0000 7fa454de6640 1 -- 172.21.15.97:0/2799566209 wait complete. 2024-03-29T04:07:34.704 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:07:34.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.824+0000 7ff25f100640 1 Processor -- start 2024-03-29T04:07:34.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.824+0000 7ff25f100640 1 -- start start 2024-03-29T04:07:34.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.825+0000 7ff25f100640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff25815ec60 0x7ff2580ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.825+0000 7ff25f100640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff2580b4340 0x7ff2580d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.825+0000 7ff25f100640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2580cf7f0 con 0x7ff25815e880 2024-03-29T04:07:34.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.825+0000 7ff25f100640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ff2580cfe00 con 0x7ff2580b4340 2024-03-29T04:07:34.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.825+0000 7ff25f100640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff2580cea70 con 0x7ff25815ec60 2024-03-29T04:07:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.825+0000 7ff257fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff25815ec60 0x7ff2580ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.825+0000 7ff25d69d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff2580b4340 0x7ff2580d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.825+0000 7ff25ce9c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff25815e880 0x7ff2580cdd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:37386/0 (socket says 172.21.15.97:37386) 2024-03-29T04:07:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.825+0000 7ff25ce9c640 1 -- 172.21.15.97:0/1447970864 learned_addr learned my addr 172.21.15.97:0/1447970864 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.826+0000 7ff257fff640 1 -- 172.21.15.97:0/1447970864 >> v2:172.21.15.154:3300/0 conn(0x7ff2580b4340 msgr2=0x7ff2580d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.826+0000 7ff257fff640 1 --2- 172.21.15.97:0/1447970864 >> v2:172.21.15.154:3300/0 conn(0x7ff2580b4340 0x7ff2580d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.826+0000 7ff257fff640 1 -- 172.21.15.97:0/1447970864 >> v1:172.21.15.154:6789/0 conn(0x7ff25815e880 legacy=0x7ff2580cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.826+0000 7ff257fff640 1 -- 172.21.15.97:0/1447970864 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2580ced50 con 0x7ff25815ec60 2024-03-29T04:07:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.826+0000 7ff257fff640 1 --2- 172.21.15.97:0/1447970864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff25815ec60 0x7ff2580ce450 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff2440097a0 tx=0x7ff24402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14193ed53a73c07c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.827+0000 7ff2577fe640 1 -- 172.21.15.97:0/1447970864 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244035020 con 0x7ff25815ec60 2024-03-29T04:07:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.827+0000 7ff2577fe640 1 -- 172.21.15.97:0/1447970864 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff244031830 con 0x7ff25815ec60 2024-03-29T04:07:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.827+0000 7ff2577fe640 1 -- 172.21.15.97:0/1447970864 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244031c00 con 0x7ff25815ec60 2024-03-29T04:07:34.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.827+0000 7ff25f100640 1 -- 172.21.15.97:0/1447970864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff25815ec60 msgr2=0x7ff2580ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.827+0000 7ff25f100640 1 --2- 172.21.15.97:0/1447970864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff25815ec60 0x7ff2580ce450 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff2440097a0 tx=0x7ff24402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.827+0000 7ff25f100640 1 -- 172.21.15.97:0/1447970864 shutdown_connections 2024-03-29T04:07:34.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.827+0000 7ff25f100640 1 --2- 172.21.15.97:0/1447970864 >> v2:172.21.15.154:3300/0 conn(0x7ff2580b4340 0x7ff2580d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.827+0000 7ff25f100640 1 --2- 172.21.15.97:0/1447970864 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff25815ec60 0x7ff2580ce450 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.827+0000 7ff25f100640 1 -- 172.21.15.97:0/1447970864 >> 172.21.15.97:0/1447970864 conn(0x7ff2580b6860 msgr2=0x7ff2580b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:34.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.827+0000 7ff25f100640 1 -- 172.21.15.97:0/1447970864 shutdown_connections 2024-03-29T04:07:34.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.827+0000 7ff25f100640 1 -- 172.21.15.97:0/1447970864 wait complete. 2024-03-29T04:07:34.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.828+0000 7ff25f100640 1 Processor -- start 2024-03-29T04:07:34.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.828+0000 7ff25f100640 1 -- start start 2024-03-29T04:07:34.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.829+0000 7ff25f100640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2580b4340 0x7ff25815cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.829+0000 7ff25f100640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff25815e880 0x7ff258160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.829+0000 7ff25f100640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff25815ec60 0x7ff25814bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.829+0000 7ff25f100640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff2580d0950 con 0x7ff2580b4340 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.829+0000 7ff25f100640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff2580d07d0 con 0x7ff25815e880 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.829+0000 7ff25f100640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff2580d0ad0 con 0x7ff25815ec60 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.829+0000 7ff25ce9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2580b4340 0x7ff25815cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.829+0000 7ff25ce9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2580b4340 0x7ff25815cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60186/0 (socket says 172.21.15.97:60186) 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.829+0000 7ff25ce9c640 1 -- 172.21.15.97:0/1381403501 learned_addr learned my addr 172.21.15.97:0/1381403501 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.829+0000 7ff25d69d640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff25815ec60 0x7ff25814bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.829+0000 7ff257fff640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff25815e880 0x7ff258160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.829+0000 7ff25ce9c640 1 -- 172.21.15.97:0/1381403501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff25815ec60 msgr2=0x7ff25814bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.829+0000 7ff25ce9c640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff25815ec60 0x7ff25814bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.830+0000 7ff25ce9c640 1 -- 172.21.15.97:0/1381403501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff25815e880 msgr2=0x7ff258160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.830+0000 7ff25ce9c640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff25815e880 0x7ff258160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:34.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.830+0000 7ff25ce9c640 1 -- 172.21.15.97:0/1381403501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25815d630 con 0x7ff2580b4340 2024-03-29T04:07:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.830+0000 7ff25d69d640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff25815ec60 0x7ff25814bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:07:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.830+0000 7ff257fff640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff25815e880 0x7ff258160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:07:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.830+0000 7ff25ce9c640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2580b4340 0x7ff25815cf30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00c400 tx=0x7ff24c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.830+0000 7ff2557fa640 1 -- 172.21.15.97:0/1381403501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c017070 con 0x7ff2580b4340 2024-03-29T04:07:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.830+0000 7ff2557fa640 1 -- 172.21.15.97:0/1381403501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff24c005ce0 con 0x7ff2580b4340 2024-03-29T04:07:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.830+0000 7ff25f100640 1 -- 172.21.15.97:0/1381403501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25815d910 con 0x7ff2580b4340 2024-03-29T04:07:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.830+0000 7ff2557fa640 1 -- 172.21.15.97:0/1381403501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c0072f0 con 0x7ff2580b4340 2024-03-29T04:07:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.830+0000 7ff25f100640 1 -- 172.21.15.97:0/1381403501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25815b880 con 0x7ff2580b4340 2024-03-29T04:07:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.831+0000 7ff25f100640 1 -- 172.21.15.97:0/1381403501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff25815b670 con 0x7ff2580b4340 2024-03-29T04:07:34.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.831+0000 7ff2557fa640 1 -- 172.21.15.97:0/1381403501 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff24c01a070 con 0x7ff2580b4340 2024-03-29T04:07:34.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.832+0000 7ff2557fa640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff23c06c530 0x7ff23c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:34.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.832+0000 7ff2557fa640 1 -- 172.21.15.97:0/1381403501 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff24c08a7a0 con 0x7ff2580b4340 2024-03-29T04:07:34.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.832+0000 7ff257fff640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff23c06c530 0x7ff23c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:34.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.832+0000 7ff257fff640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff23c06c530 0x7ff23c06e9f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff244036000 tx=0x7ff244008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:34.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:34.836+0000 7ff2557fa640 1 -- 172.21.15.97:0/1381403501 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff24c059110 con 0x7ff2580b4340 2024-03-29T04:07:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.047+0000 7ff25f100640 1 -- 172.21.15.97:0/1381403501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff258068640 con 0x7ff2580b4340 2024-03-29T04:07:35.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.048+0000 7ff2557fa640 1 -- 172.21.15.97:0/1381403501 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7ff24c058a10 con 0x7ff2580b4340 2024-03-29T04:07:35.028 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:35.028 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3767188750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3767188750},{"type":"v1","addr":"172.21.15.97:6835","nonce":3767188750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-29T04:07:35.028 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 2 2024-03-29T04:07:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.050+0000 7ff25f100640 1 -- 172.21.15.97:0/1381403501 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff23c06c530 msgr2=0x7ff23c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.050+0000 7ff25f100640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff23c06c530 0x7ff23c06e9f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff244036000 tx=0x7ff244008030 comp rx=0 tx=0).stop 2024-03-29T04:07:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.050+0000 7ff25f100640 1 -- 172.21.15.97:0/1381403501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2580b4340 msgr2=0x7ff25815cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.050+0000 7ff25f100640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2580b4340 0x7ff25815cf30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00c400 tx=0x7ff24c00c8c0 comp rx=0 tx=0).stop 2024-03-29T04:07:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.050+0000 7ff25f100640 1 -- 172.21.15.97:0/1381403501 shutdown_connections 2024-03-29T04:07:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.050+0000 7ff25f100640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff23c06c530 0x7ff23c06e9f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.050+0000 7ff25f100640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff25815ec60 0x7ff25814bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.050+0000 7ff25f100640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff25815e880 0x7ff258160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.050+0000 7ff25f100640 1 --2- 172.21.15.97:0/1381403501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2580b4340 0x7ff25815cf30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:35.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.050+0000 7ff25f100640 1 -- 172.21.15.97:0/1381403501 >> 172.21.15.97:0/1381403501 conn(0x7ff2580b6860 msgr2=0x7ff2580d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:35.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.050+0000 7ff25f100640 1 -- 172.21.15.97:0/1381403501 shutdown_connections 2024-03-29T04:07:35.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:35.050+0000 7ff25f100640 1 -- 172.21.15.97:0/1381403501 wait complete. 2024-03-29T04:07:37.041 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:07:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.173+0000 7fea354d9640 1 Processor -- start 2024-03-29T04:07:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.173+0000 7fea354d9640 1 -- start start 2024-03-29T04:07:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea354d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3015e930 0x7fea300cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea354d9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea300cf670 0x7fea300d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea354d9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fea300b3c50 0x7fea300d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea354d9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fea3006c780 con 0x7fea300cf670 2024-03-29T04:07:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea354d9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fea300cfb30 con 0x7fea300b3c50 2024-03-29T04:07:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea354d9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fea300cfe10 con 0x7fea3015e930 2024-03-29T04:07:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea2e7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea300cf670 0x7fea300d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea2e7fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea300cf670 0x7fea300d2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60198/0 (socket says 172.21.15.97:60198) 2024-03-29T04:07:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea2e7fc640 1 -- 172.21.15.97:0/1135875175 learned_addr learned my addr 172.21.15.97:0/1135875175 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea2e7fc640 1 -- 172.21.15.97:0/1135875175 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3015e930 msgr2=0x7fea300cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea2f7fe640 1 --2- 172.21.15.97:0/1135875175 >> v2:172.21.15.154:3300/0 conn(0x7fea300b3c50 0x7fea300d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea2e7fc640 1 --2- 172.21.15.97:0/1135875175 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3015e930 0x7fea300cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea2e7fc640 1 -- 172.21.15.97:0/1135875175 >> v2:172.21.15.154:3300/0 conn(0x7fea300b3c50 msgr2=0x7fea300d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea2e7fc640 1 --2- 172.21.15.97:0/1135875175 >> v2:172.21.15.154:3300/0 conn(0x7fea300b3c50 0x7fea300d3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.174+0000 7fea2e7fc640 1 -- 172.21.15.97:0/1135875175 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea300d00f0 con 0x7fea300cf670 2024-03-29T04:07:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.175+0000 7fea2e7fc640 1 --2- 172.21.15.97:0/1135875175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea300cf670 0x7fea300d2c00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea300ba6a0 tx=0x7fea2402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35bc6562f080b9d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.175+0000 7fea2dffb640 1 -- 172.21.15.97:0/1135875175 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2402e070 con 0x7fea300cf670 2024-03-29T04:07:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.175+0000 7fea2dffb640 1 -- 172.21.15.97:0/1135875175 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea24037070 con 0x7fea300cf670 2024-03-29T04:07:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.175+0000 7fea2dffb640 1 -- 172.21.15.97:0/1135875175 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24031a00 con 0x7fea300cf670 2024-03-29T04:07:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.176+0000 7fea354d9640 1 -- 172.21.15.97:0/1135875175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea300cf670 msgr2=0x7fea300d2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.176+0000 7fea354d9640 1 --2- 172.21.15.97:0/1135875175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea300cf670 0x7fea300d2c00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea300ba6a0 tx=0x7fea2402b880 comp rx=0 tx=0).stop 2024-03-29T04:07:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.176+0000 7fea354d9640 1 -- 172.21.15.97:0/1135875175 shutdown_connections 2024-03-29T04:07:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.176+0000 7fea354d9640 1 --2- 172.21.15.97:0/1135875175 >> v2:172.21.15.154:3300/0 conn(0x7fea300b3c50 0x7fea300d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.176+0000 7fea354d9640 1 --2- 172.21.15.97:0/1135875175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea300cf670 0x7fea300d2c00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.176+0000 7fea354d9640 1 --2- 172.21.15.97:0/1135875175 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea3015e930 0x7fea300cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.176+0000 7fea354d9640 1 -- 172.21.15.97:0/1135875175 >> 172.21.15.97:0/1135875175 conn(0x7fea3008bfc0 msgr2=0x7fea300d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.176+0000 7fea354d9640 1 -- 172.21.15.97:0/1135875175 shutdown_connections 2024-03-29T04:07:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.176+0000 7fea354d9640 1 -- 172.21.15.97:0/1135875175 wait complete. 2024-03-29T04:07:37.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.177+0000 7fea354d9640 1 Processor -- start 2024-03-29T04:07:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.177+0000 7fea354d9640 1 -- start start 2024-03-29T04:07:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea354d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea300b3c50 0x7fea3015a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea354d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea300cf670 0x7fea30156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea354d9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea3015e930 0x7fea30157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea354d9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea3015ee90 con 0x7fea300cf670 2024-03-29T04:07:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea354d9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fea3015ed10 con 0x7fea3015e930 2024-03-29T04:07:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea354d9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fea300d0c50 con 0x7fea300b3c50 2024-03-29T04:07:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea2e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea300cf670 0x7fea30156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea2e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea300cf670 0x7fea30156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60208/0 (socket says 172.21.15.97:60208) 2024-03-29T04:07:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea2e7fc640 1 -- 172.21.15.97:0/1589668 learned_addr learned my addr 172.21.15.97:0/1589668 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea2effd640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea300b3c50 0x7fea3015a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea2f7fe640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea3015e930 0x7fea30157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea2e7fc640 1 -- 172.21.15.97:0/1589668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea300b3c50 msgr2=0x7fea3015a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea2e7fc640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea300b3c50 0x7fea3015a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea2e7fc640 1 -- 172.21.15.97:0/1589668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea3015e930 msgr2=0x7fea30157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea2e7fc640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea3015e930 0x7fea30157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.178+0000 7fea2e7fc640 1 -- 172.21.15.97:0/1589668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3015b1e0 con 0x7fea300cf670 2024-03-29T04:07:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.179+0000 7fea2e7fc640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea300cf670 0x7fea30156dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea2402b8c0 tx=0x7fea240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.179+0000 7fea177fe640 1 -- 172.21.15.97:0/1589668 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24032880 con 0x7fea300cf670 2024-03-29T04:07:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.179+0000 7fea177fe640 1 -- 172.21.15.97:0/1589668 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea24032a20 con 0x7fea300cf670 2024-03-29T04:07:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.179+0000 7fea354d9640 1 -- 172.21.15.97:0/1589668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea30160680 con 0x7fea300cf670 2024-03-29T04:07:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.179+0000 7fea177fe640 1 -- 172.21.15.97:0/1589668 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24032e10 con 0x7fea300cf670 2024-03-29T04:07:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.179+0000 7fea354d9640 1 -- 172.21.15.97:0/1589668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea30160c40 con 0x7fea300cf670 2024-03-29T04:07:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.180+0000 7fea354d9640 1 -- 172.21.15.97:0/1589668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea30068640 con 0x7fea300cf670 2024-03-29T04:07:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.180+0000 7fea177fe640 1 -- 172.21.15.97:0/1589668 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fea2404c060 con 0x7fea300cf670 2024-03-29T04:07:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.181+0000 7fea177fe640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fea1c06c530 0x7fea1c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.181+0000 7fea177fe640 1 -- 172.21.15.97:0/1589668 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fea240acfc0 con 0x7fea300cf670 2024-03-29T04:07:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.181+0000 7fea2effd640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fea1c06c530 0x7fea1c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.181+0000 7fea2effd640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fea1c06c530 0x7fea1c06e9f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea300b40c0 tx=0x7fea20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.185+0000 7fea177fe640 1 -- 172.21.15.97:0/1589668 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fea24040070 con 0x7fea300cf670 2024-03-29T04:07:37.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.396+0000 7fea354d9640 1 -- 172.21.15.97:0/1589668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea300d4be0 con 0x7fea300cf670 2024-03-29T04:07:37.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.398+0000 7fea177fe640 1 -- 172.21.15.97:0/1589668 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7fea2407b930 con 0x7fea300cf670 2024-03-29T04:07:37.377 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:37.378 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1127043830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127043830},{"type":"v1","addr":"172.21.15.154:6834","nonce":1127043830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3767188750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3767188750},{"type":"v1","addr":"172.21.15.97:6835","nonce":3767188750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3651340478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3651340478},{"type":"v1","addr":"172.21.15.97:6837","nonce":3651340478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3669418265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3669418265},{"type":"v1","addr":"172.21.15.154:6835","nonce":3669418265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-29T04:07:37.378 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 3 2024-03-29T04:07:37.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.400+0000 7fea354d9640 1 -- 172.21.15.97:0/1589668 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fea1c06c530 msgr2=0x7fea1c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.400+0000 7fea354d9640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fea1c06c530 0x7fea1c06e9f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea300b40c0 tx=0x7fea20008040 comp rx=0 tx=0).stop 2024-03-29T04:07:37.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.400+0000 7fea354d9640 1 -- 172.21.15.97:0/1589668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea300cf670 msgr2=0x7fea30156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.400+0000 7fea354d9640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea300cf670 0x7fea30156dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea2402b8c0 tx=0x7fea240046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.400+0000 7fea354d9640 1 -- 172.21.15.97:0/1589668 shutdown_connections 2024-03-29T04:07:37.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.400+0000 7fea354d9640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fea1c06c530 0x7fea1c06e9f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.400+0000 7fea354d9640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea3015e930 0x7fea30157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.400+0000 7fea354d9640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea300cf670 0x7fea30156dc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.400+0000 7fea354d9640 1 --2- 172.21.15.97:0/1589668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fea300b3c50 0x7fea3015a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.400+0000 7fea354d9640 1 -- 172.21.15.97:0/1589668 >> 172.21.15.97:0/1589668 conn(0x7fea3008bfc0 msgr2=0x7fea300d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:37.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.400+0000 7fea354d9640 1 -- 172.21.15.97:0/1589668 shutdown_connections 2024-03-29T04:07:37.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.400+0000 7fea354d9640 1 -- 172.21.15.97:0/1589668 wait complete. 2024-03-29T04:07:37.390 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:37.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.511+0000 7f5c7b11c640 1 Processor -- start 2024-03-29T04:07:37.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.511+0000 7f5c7b11c640 1 -- start start 2024-03-29T04:07:37.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.511+0000 7f5c7b11c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c7415e6e0 0x7f5c7409b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.511+0000 7f5c7b11c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c740d0660 0x7f5c741623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.511+0000 7f5c7b11c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c740d1a10 con 0x7f5c741628e0 2024-03-29T04:07:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.511+0000 7f5c7b11c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f5c740d0e90 con 0x7f5c7415e6e0 2024-03-29T04:07:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.511+0000 7f5c7b11c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5c740d1170 con 0x7f5c740d0660 2024-03-29T04:07:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c796b9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5c741628e0 0x7f5c740b30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52634/0 (socket says 172.21.15.97:52634) 2024-03-29T04:07:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c796b9640 1 -- 172.21.15.97:0/721085591 learned_addr learned my addr 172.21.15.97:0/721085591 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1072626061 0 0) 0x7f5c740d1a10 con 0x7f5c741628e0 2024-03-29T04:07:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c73fff640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c740d0660 0x7f5c741623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c78eb8640 1 --2- 172.21.15.97:0/721085591 >> v2:172.21.15.154:3300/0 conn(0x7f5c7415e6e0 0x7f5c7409b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c64003520 con 0x7f5c741628e0 2024-03-29T04:07:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 850145541 0 0) 0x7f5c64003520 con 0x7f5c741628e0 2024-03-29T04:07:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c740d0660 msgr2=0x7f5c741623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:37.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c740d0660 0x7f5c741623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 >> v2:172.21.15.154:3300/0 conn(0x7f5c7415e6e0 msgr2=0x7f5c7409b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 --2- 172.21.15.97:0/721085591 >> v2:172.21.15.154:3300/0 conn(0x7f5c7415e6e0 0x7f5c7409b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c740d1450 con 0x7f5c741628e0 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f5c6c0019f0 con 0x7f5c741628e0 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 >> v1:172.21.15.97:6789/0 conn(0x7f5c741628e0 legacy=0x7f5c740b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c740d0660 0x7f5c640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64003f30 0x7f5c64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c64004d40 0x7f5c640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5c740d0e90 con 0x7f5c64003f30 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5c740d1170 con 0x7f5c64004d40 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6400b9c0 con 0x7f5c740d0660 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c73fff640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64003f30 0x7f5c64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c78eb8640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c740d0660 0x7f5c640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.512+0000 7f5c796b9640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c64004d40 0x7f5c640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.513+0000 7f5c73fff640 1 -- 172.21.15.97:0/721085591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c740d0660 msgr2=0x7f5c640039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:37.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.513+0000 7f5c73fff640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c740d0660 0x7f5c640039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.513+0000 7f5c73fff640 1 -- 172.21.15.97:0/721085591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c64004d40 msgr2=0x7f5c640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.513+0000 7f5c73fff640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c64004d40 0x7f5c640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.513+0000 7f5c73fff640 1 -- 172.21.15.97:0/721085591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c64003520 con 0x7f5c64003f30 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.513+0000 7f5c796b9640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c64004d40 0x7f5c640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.513+0000 7f5c78eb8640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c740d0660 0x7f5c640039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.513+0000 7f5c73fff640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64003f30 0x7f5c64004800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009ac0 tx=0x7f5c64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58281a9681341407 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.513+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60039070 con 0x7f5c64003f30 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.513+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6002c6b0 con 0x7f5c64003f30 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.513+0000 7f5c737fe640 1 -- 172.21.15.97:0/721085591 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60009690 con 0x7f5c64003f30 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.514+0000 7f5c7b11c640 1 -- 172.21.15.97:0/721085591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64003f30 msgr2=0x7f5c64004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.514+0000 7f5c7b11c640 1 --2- 172.21.15.97:0/721085591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c64003f30 0x7f5c64004800 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009ac0 tx=0x7f5c64005830 comp rx=0 tx=0).stop 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.514+0000 7f5c796b9640 1 -- 172.21.15.97:0/721085591 reap_dead start 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.514+0000 7f5c7b11c640 1 -- 172.21.15.97:0/721085591 shutdown_connections 2024-03-29T04:07:37.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.514+0000 7f5c7b11c640 1 -- 172.21.15.97:0/721085591 >> 172.21.15.97:0/721085591 conn(0x7f5c740b3f10 msgr2=0x7f5c740b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:37.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.514+0000 7f5c7b11c640 1 -- 172.21.15.97:0/721085591 shutdown_connections 2024-03-29T04:07:37.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.514+0000 7f5c7b11c640 1 -- 172.21.15.97:0/721085591 wait complete. 2024-03-29T04:07:37.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.515+0000 7f5c7b11c640 1 Processor -- start 2024-03-29T04:07:37.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.515+0000 7f5c7b11c640 1 -- start start 2024-03-29T04:07:37.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c7b11c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c64003f30 0x7f5c7414f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c7b11c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c740b0c20 0x7f5c740af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c7b11c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7414bea0 0x7f5c740b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c7b11c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7406d910 con 0x7f5c740b0c20 2024-03-29T04:07:37.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c7b11c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5c740b1d80 con 0x7f5c7414bea0 2024-03-29T04:07:37.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c7b11c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5c740af950 con 0x7f5c64003f30 2024-03-29T04:07:37.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c73fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c740b0c20 0x7f5c740af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c73fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c740b0c20 0x7f5c740af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60220/0 (socket says 172.21.15.97:60220) 2024-03-29T04:07:37.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c73fff640 1 -- 172.21.15.97:0/3210234153 learned_addr learned my addr 172.21.15.97:0/3210234153 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:37.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c78eb8640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c64003f30 0x7f5c7414f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c796b9640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7414bea0 0x7f5c740b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c73fff640 1 -- 172.21.15.97:0/3210234153 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c64003f30 msgr2=0x7f5c7414f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c73fff640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c64003f30 0x7f5c7414f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c73fff640 1 -- 172.21.15.97:0/3210234153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7414bea0 msgr2=0x7f5c740b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c73fff640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7414bea0 0x7f5c740b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c73fff640 1 -- 172.21.15.97:0/3210234153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c740afb80 con 0x7f5c740b0c20 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c796b9640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7414bea0 0x7f5c740b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.516+0000 7f5c73fff640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c740b0c20 0x7f5c740af410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5c60037040 tx=0x7f5c60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.517+0000 7f5c717fa640 1 -- 172.21.15.97:0/3210234153 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60039050 con 0x7f5c740b0c20 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.517+0000 7f5c717fa640 1 -- 172.21.15.97:0/3210234153 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c60045070 con 0x7f5c740b0c20 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.517+0000 7f5c7b11c640 1 -- 172.21.15.97:0/3210234153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c740afe60 con 0x7f5c740b0c20 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.517+0000 7f5c717fa640 1 -- 172.21.15.97:0/3210234153 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c600044e0 con 0x7f5c740b0c20 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.517+0000 7f5c7b11c640 1 -- 172.21.15.97:0/3210234153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c740b0420 con 0x7f5c740b0c20 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.518+0000 7f5c7b11c640 1 -- 172.21.15.97:0/3210234153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c74068640 con 0x7f5c740b0c20 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.518+0000 7f5c717fa640 1 -- 172.21.15.97:0/3210234153 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5c60003810 con 0x7f5c740b0c20 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.519+0000 7f5c717fa640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5c5806c560 0x7f5c5806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.519+0000 7f5c78eb8640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5c5806c560 0x7f5c5806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.519+0000 7f5c717fa640 1 -- 172.21.15.97:0/3210234153 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f5c600ad450 con 0x7f5c740b0c20 2024-03-29T04:07:37.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.519+0000 7f5c78eb8640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5c5806c560 0x7f5c5806ea20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009bc0 tx=0x7f5c68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:37.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.523+0000 7f5c717fa640 1 -- 172.21.15.97:0/3210234153 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5c6007b800 con 0x7f5c740b0c20 2024-03-29T04:07:37.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.701+0000 7f5c7b11c640 1 -- 172.21.15.97:0/3210234153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c740d1e40 con 0x7f5c740b0c20 2024-03-29T04:07:37.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.702+0000 7f5c717fa640 1 -- 172.21.15.97:0/3210234153 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7f5c6007fb00 con 0x7f5c740b0c20 2024-03-29T04:07:37.682 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:37.682 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:30.780150+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:37.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.704+0000 7f5c7b11c640 1 -- 172.21.15.97:0/3210234153 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5c5806c560 msgr2=0x7f5c5806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.704+0000 7f5c7b11c640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5c5806c560 0x7f5c5806ea20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009bc0 tx=0x7f5c68008040 comp rx=0 tx=0).stop 2024-03-29T04:07:37.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.704+0000 7f5c7b11c640 1 -- 172.21.15.97:0/3210234153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c740b0c20 msgr2=0x7f5c740af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.704+0000 7f5c7b11c640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c740b0c20 0x7f5c740af410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5c60037040 tx=0x7f5c60004730 comp rx=0 tx=0).stop 2024-03-29T04:07:37.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.704+0000 7f5c7b11c640 1 -- 172.21.15.97:0/3210234153 shutdown_connections 2024-03-29T04:07:37.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.704+0000 7f5c7b11c640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5c5806c560 0x7f5c5806ea20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.704+0000 7f5c7b11c640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c7414bea0 0x7f5c740b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.704+0000 7f5c7b11c640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c740b0c20 0x7f5c740af410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.704+0000 7f5c7b11c640 1 --2- 172.21.15.97:0/3210234153 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c64003f30 0x7f5c7414f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.704+0000 7f5c7b11c640 1 -- 172.21.15.97:0/3210234153 >> 172.21.15.97:0/3210234153 conn(0x7f5c740b3f10 msgr2=0x7f5c7415c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:37.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.704+0000 7f5c7b11c640 1 -- 172.21.15.97:0/3210234153 shutdown_connections 2024-03-29T04:07:37.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.704+0000 7f5c7b11c640 1 -- 172.21.15.97:0/3210234153 wait complete. 2024-03-29T04:07:37.694 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:07:37.695 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:07:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.817+0000 7f9818d1c640 1 Processor -- start 2024-03-29T04:07:37.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.817+0000 7f9818d1c640 1 -- start start 2024-03-29T04:07:37.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.818+0000 7f9818d1c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f98140cfde0 0x7f98140d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.818+0000 7f9818d1c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f981415ed00 con 0x7f98140d0f90 2024-03-29T04:07:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.818+0000 7f9818d1c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98140d21e0 con 0x7f981415e920 2024-03-29T04:07:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.818+0000 7f9818d1c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f98140d2480 con 0x7f98140cfde0 2024-03-29T04:07:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.818+0000 7f9812d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f98140d0f90 0x7f981406c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52638/0 (socket says 172.21.15.97:52638) 2024-03-29T04:07:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f9812d9d640 1 -- 172.21.15.97:0/1838830135 learned_addr learned my addr 172.21.15.97:0/1838830135 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f9811d9b640 1 --2- 172.21.15.97:0/1838830135 >> v2:172.21.15.154:3300/0 conn(0x7f98140cfde0 0x7f98140d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3665297086 0 0) 0x7f981415ed00 con 0x7f98140d0f90 2024-03-29T04:07:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97fc003520 con 0x7f98140d0f90 2024-03-29T04:07:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3367765026 0 0) 0x7f97fc003520 con 0x7f98140d0f90 2024-03-29T04:07:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 >> v2:172.21.15.154:3300/0 conn(0x7f98140cfde0 msgr2=0x7f98140d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 --2- 172.21.15.97:0/1838830135 >> v2:172.21.15.154:3300/0 conn(0x7f98140cfde0 0x7f98140d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 >> v1:172.21.15.154:6789/0 conn(0x7f981415e920 legacy=0x7f98140d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98140d2760 con 0x7f98140d0f90 2024-03-29T04:07:37.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f980c001670 con 0x7f98140d0f90 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 >> v1:172.21.15.97:6789/0 conn(0x7f98140d0f90 legacy=0x7f981406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 --2- 172.21.15.97:0/1838830135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97fc0039b0 0x7f97fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 --2- 172.21.15.97:0/1838830135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97fc004800 0x7f97fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 --2- 172.21.15.97:0/1838830135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97fc0051c0 0x7f97fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f98140d2480 con 0x7f97fc0039b0 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc00c060 con 0x7f97fc0051c0 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f97fc00c260 con 0x7f97fc004800 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981259c640 1 --2- 172.21.15.97:0/1838830135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97fc0039b0 0x7f97fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f9812d9d640 1 --2- 172.21.15.97:0/1838830135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97fc0051c0 0x7f97fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f9811d9b640 1 --2- 172.21.15.97:0/1838830135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97fc004800 0x7f97fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981259c640 1 -- 172.21.15.97:0/1838830135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97fc004800 msgr2=0x7f97fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981259c640 1 --2- 172.21.15.97:0/1838830135 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97fc004800 0x7f97fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981259c640 1 -- 172.21.15.97:0/1838830135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97fc0051c0 msgr2=0x7f97fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:37.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981259c640 1 --2- 172.21.15.97:0/1838830135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97fc0051c0 0x7f97fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f981259c640 1 -- 172.21.15.97:0/1838830135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc003520 con 0x7f97fc0039b0 2024-03-29T04:07:37.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.819+0000 7f9812d9d640 1 -- 172.21.15.97:0/1838830135 reap_dead start 2024-03-29T04:07:37.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.820+0000 7f981259c640 1 --2- 172.21.15.97:0/1838830135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97fc0039b0 0x7f97fc003dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9804001e50 tx=0x7f97fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32ec55d9a869a1a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:37.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.820+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804001a70 con 0x7f97fc0039b0 2024-03-29T04:07:37.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.820+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9804003b70 con 0x7f97fc0039b0 2024-03-29T04:07:37.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.820+0000 7f981159a640 1 -- 172.21.15.97:0/1838830135 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98040050d0 con 0x7f97fc0039b0 2024-03-29T04:07:37.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.821+0000 7f9818d1c640 1 -- 172.21.15.97:0/1838830135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97fc0039b0 msgr2=0x7f97fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.821+0000 7f9818d1c640 1 --2- 172.21.15.97:0/1838830135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97fc0039b0 0x7f97fc003dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9804001e50 tx=0x7f97fc005830 comp rx=0 tx=0).stop 2024-03-29T04:07:37.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.821+0000 7f9818d1c640 1 -- 172.21.15.97:0/1838830135 shutdown_connections 2024-03-29T04:07:37.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.821+0000 7f9818d1c640 1 --2- 172.21.15.97:0/1838830135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97fc0039b0 0x7f97fc003dd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.821+0000 7f9818d1c640 1 -- 172.21.15.97:0/1838830135 >> 172.21.15.97:0/1838830135 conn(0x7f981408bfc0 msgr2=0x7f98140d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:37.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.821+0000 7f9818d1c640 1 -- 172.21.15.97:0/1838830135 shutdown_connections 2024-03-29T04:07:37.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.821+0000 7f9818d1c640 1 -- 172.21.15.97:0/1838830135 wait complete. 2024-03-29T04:07:37.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.822+0000 7f9818d1c640 1 Processor -- start 2024-03-29T04:07:37.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.822+0000 7f9818d1c640 1 -- start start 2024-03-29T04:07:37.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9818d1c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97fc0039b0 0x7f9814159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9818d1c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98140b7ba0 0x7f98141610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9818d1c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9814161600 0x7f98141f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9818d1c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f98140d4ef0 con 0x7f98140b7ba0 2024-03-29T04:07:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9818d1c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f98140d2d20 con 0x7f9814161600 2024-03-29T04:07:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9818d1c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f98140d5070 con 0x7f97fc0039b0 2024-03-29T04:07:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9811d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98140b7ba0 0x7f98141610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9811d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98140b7ba0 0x7f98141610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60234/0 (socket says 172.21.15.97:60234) 2024-03-29T04:07:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9811d9b640 1 -- 172.21.15.97:0/2329975767 learned_addr learned my addr 172.21.15.97:0/2329975767 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9812d9d640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9814161600 0x7f98141f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f981259c640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97fc0039b0 0x7f9814159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9811d9b640 1 -- 172.21.15.97:0/2329975767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97fc0039b0 msgr2=0x7f9814159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9811d9b640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97fc0039b0 0x7f9814159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9811d9b640 1 -- 172.21.15.97:0/2329975767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9814161600 msgr2=0x7f98141f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9811d9b640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9814161600 0x7f98141f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9811d9b640 1 -- 172.21.15.97:0/2329975767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98140b8d60 con 0x7f98140b7ba0 2024-03-29T04:07:37.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9812d9d640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9814161600 0x7f98141f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:07:37.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.823+0000 7f9811d9b640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98140b7ba0 0x7f98141610c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f98080119c0 tx=0x7f9808011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:37.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.824+0000 7f9802ffd640 1 -- 172.21.15.97:0/2329975767 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980800fb50 con 0x7f98140b7ba0 2024-03-29T04:07:37.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.824+0000 7f9802ffd640 1 -- 172.21.15.97:0/2329975767 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f980800fcf0 con 0x7f98140b7ba0 2024-03-29T04:07:37.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.824+0000 7f9818d1c640 1 -- 172.21.15.97:0/2329975767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98140b9040 con 0x7f98140b7ba0 2024-03-29T04:07:37.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.824+0000 7f9802ffd640 1 -- 172.21.15.97:0/2329975767 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980800fea0 con 0x7f98140b7ba0 2024-03-29T04:07:37.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.824+0000 7f9818d1c640 1 -- 172.21.15.97:0/2329975767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98140b45a0 con 0x7f98140b7ba0 2024-03-29T04:07:37.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.825+0000 7f9818d1c640 1 -- 172.21.15.97:0/2329975767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98140a2560 con 0x7f98140b7ba0 2024-03-29T04:07:37.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.825+0000 7f9802ffd640 1 -- 172.21.15.97:0/2329975767 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9808012040 con 0x7f98140b7ba0 2024-03-29T04:07:37.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.825+0000 7f9802ffd640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f97f406c530 0x7f97f406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:37.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.825+0000 7f9802ffd640 1 -- 172.21.15.97:0/2329975767 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f980808b920 con 0x7f98140b7ba0 2024-03-29T04:07:37.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.826+0000 7f981259c640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f97f406c530 0x7f97f406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:37.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.826+0000 7f981259c640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f97f406c530 0x7f97f406e9f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9804034000 tx=0x7f9804009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:37.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:37.830+0000 7f9802ffd640 1 -- 172.21.15.97:0/2329975767 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9808019030 con 0x7f98140b7ba0 2024-03-29T04:07:37.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.003+0000 7f9818d1c640 1 -- 172.21.15.97:0/2329975767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98141619e0 con 0x7f98140b7ba0 2024-03-29T04:07:37.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.004+0000 7f9802ffd640 1 -- 172.21.15.97:0/2329975767 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f980805a320 con 0x7f98140b7ba0 2024-03-29T04:07:37.984 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:07:37.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.006+0000 7f9818d1c640 1 -- 172.21.15.97:0/2329975767 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f97f406c530 msgr2=0x7f97f406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.006+0000 7f9818d1c640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f97f406c530 0x7f97f406e9f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9804034000 tx=0x7f9804009030 comp rx=0 tx=0).stop 2024-03-29T04:07:37.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.006+0000 7f9818d1c640 1 -- 172.21.15.97:0/2329975767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98140b7ba0 msgr2=0x7f98141610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:37.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.006+0000 7f9818d1c640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98140b7ba0 0x7f98141610c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f98080119c0 tx=0x7f9808011e80 comp rx=0 tx=0).stop 2024-03-29T04:07:37.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.006+0000 7f9818d1c640 1 -- 172.21.15.97:0/2329975767 shutdown_connections 2024-03-29T04:07:37.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.006+0000 7f9818d1c640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f97f406c530 0x7f97f406e9f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.006+0000 7f9818d1c640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9814161600 0x7f98141f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.006+0000 7f9818d1c640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98140b7ba0 0x7f98141610c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.006+0000 7f9818d1c640 1 --2- 172.21.15.97:0/2329975767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97fc0039b0 0x7f9814159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:37.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.006+0000 7f9818d1c640 1 -- 172.21.15.97:0/2329975767 >> 172.21.15.97:0/2329975767 conn(0x7f981408bfc0 msgr2=0x7f98140b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:37.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.006+0000 7f9818d1c640 1 -- 172.21.15.97:0/2329975767 shutdown_connections 2024-03-29T04:07:37.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.006+0000 7f9818d1c640 1 -- 172.21.15.97:0/2329975767 wait complete. 2024-03-29T04:07:37.996 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T04:07:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.117+0000 7f0890bf4640 1 Processor -- start 2024-03-29T04:07:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.117+0000 7f0890bf4640 1 -- start start 2024-03-29T04:07:38.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.118+0000 7f0890bf4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f088c15e920 0x7f088c0d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.118+0000 7f0890bf4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c0cfde0 0x7f088c0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.118+0000 7f0890bf4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f088c0d3780 con 0x7f088c0d0f90 2024-03-29T04:07:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.118+0000 7f0890bf4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f088c15ed00 con 0x7f088c15e920 2024-03-29T04:07:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.118+0000 7f0890bf4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f088c0d21e0 con 0x7f088c0cfde0 2024-03-29T04:07:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.119+0000 7f088a59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f088c15e920 0x7f088c0d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.119+0000 7f0889d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c0cfde0 0x7f088c0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.119+0000 7f088ad9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f088c0d0f90 0x7f088c06c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:37402/0 (socket says 172.21.15.97:37402) 2024-03-29T04:07:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.119+0000 7f088ad9d640 1 -- 172.21.15.97:0/2497576626 learned_addr learned my addr 172.21.15.97:0/2497576626 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.119+0000 7f088a59c640 1 -- 172.21.15.97:0/2497576626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c0cfde0 msgr2=0x7f088c0d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.119+0000 7f088a59c640 1 --2- 172.21.15.97:0/2497576626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c0cfde0 0x7f088c0d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.119+0000 7f088a59c640 1 -- 172.21.15.97:0/2497576626 >> v1:172.21.15.154:6789/0 conn(0x7f088c0d0f90 legacy=0x7f088c06c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.119+0000 7f088a59c640 1 -- 172.21.15.97:0/2497576626 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c0d2480 con 0x7f088c15e920 2024-03-29T04:07:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.119+0000 7f088959a640 1 -- 172.21.15.97:0/2497576626 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4060361155 0 0) 0x7f0884001110 con 0x7f088c0d0f90 2024-03-29T04:07:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.120+0000 7f088a59c640 1 --2- 172.21.15.97:0/2497576626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f088c15e920 0x7f088c0d3240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f087c009dc0 tx=0x7f087c02be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbc144192bb989a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.120+0000 7f088959a640 1 -- 172.21.15.97:0/2497576626 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c039070 con 0x7f088c15e920 2024-03-29T04:07:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.120+0000 7f088959a640 1 -- 172.21.15.97:0/2497576626 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f087c02cc40 con 0x7f088c15e920 2024-03-29T04:07:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.120+0000 7f088959a640 1 -- 172.21.15.97:0/2497576626 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c034440 con 0x7f088c15e920 2024-03-29T04:07:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.120+0000 7f0890bf4640 1 -- 172.21.15.97:0/2497576626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f088c15e920 msgr2=0x7f088c0d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.120+0000 7f0890bf4640 1 --2- 172.21.15.97:0/2497576626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f088c15e920 0x7f088c0d3240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f087c009dc0 tx=0x7f087c02be90 comp rx=0 tx=0).stop 2024-03-29T04:07:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.120+0000 7f0890bf4640 1 -- 172.21.15.97:0/2497576626 shutdown_connections 2024-03-29T04:07:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.120+0000 7f0890bf4640 1 --2- 172.21.15.97:0/2497576626 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c0cfde0 0x7f088c0d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.120+0000 7f0890bf4640 1 --2- 172.21.15.97:0/2497576626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f088c15e920 0x7f088c0d3240 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.120+0000 7f0890bf4640 1 -- 172.21.15.97:0/2497576626 >> 172.21.15.97:0/2497576626 conn(0x7f088c08bfc0 msgr2=0x7f088c0d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.120+0000 7f0890bf4640 1 -- 172.21.15.97:0/2497576626 shutdown_connections 2024-03-29T04:07:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.121+0000 7f0890bf4640 1 -- 172.21.15.97:0/2497576626 wait complete. 2024-03-29T04:07:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.122+0000 7f0890bf4640 1 Processor -- start 2024-03-29T04:07:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.122+0000 7f0890bf4640 1 -- start start 2024-03-29T04:07:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.122+0000 7f0890bf4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c0cfde0 0x7f088c158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.122+0000 7f0890bf4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c0d0f90 0x7f088c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.122+0000 7f0890bf4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f088c15e920 0x7f088c1f8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.122+0000 7f0890bf4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f088c0d2d20 con 0x7f088c0d0f90 2024-03-29T04:07:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.122+0000 7f0890bf4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f088c0d2a40 con 0x7f088c15e920 2024-03-29T04:07:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.122+0000 7f0890bf4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f088c0d4ef0 con 0x7f088c0cfde0 2024-03-29T04:07:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f0889d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c0d0f90 0x7f088c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f0889d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c0d0f90 0x7f088c15ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60238/0 (socket says 172.21.15.97:60238) 2024-03-29T04:07:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f0889d9b640 1 -- 172.21.15.97:0/3568864580 learned_addr learned my addr 172.21.15.97:0/3568864580 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f088a59c640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c0cfde0 0x7f088c158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f088ad9d640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f088c15e920 0x7f088c1f8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f0889d9b640 1 -- 172.21.15.97:0/3568864580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c0cfde0 msgr2=0x7f088c158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f0889d9b640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c0cfde0 0x7f088c158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f0889d9b640 1 -- 172.21.15.97:0/3568864580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f088c15e920 msgr2=0x7f088c1f8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f0889d9b640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f088c15e920 0x7f088c1f8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f0889d9b640 1 -- 172.21.15.97:0/3568864580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c15b0c0 con 0x7f088c0d0f90 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f0889d9b640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c0d0f90 0x7f088c15ab80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f088000e870 tx=0x7f088000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f087affd640 1 -- 172.21.15.97:0/3568864580 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880017070 con 0x7f088c0d0f90 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f087affd640 1 -- 172.21.15.97:0/3568864580 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f088000c940 con 0x7f088c0d0f90 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f0890bf4640 1 -- 172.21.15.97:0/3568864580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088c15b370 con 0x7f088c0d0f90 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f087affd640 1 -- 172.21.15.97:0/3568864580 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088000cd30 con 0x7f088c0d0f90 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.123+0000 7f0890bf4640 1 -- 172.21.15.97:0/3568864580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088c15f8d0 con 0x7f088c0d0f90 2024-03-29T04:07:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.124+0000 7f0890bf4640 1 -- 172.21.15.97:0/3568864580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088c068640 con 0x7f088c0d0f90 2024-03-29T04:07:38.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.125+0000 7f087affd640 1 -- 172.21.15.97:0/3568864580 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f088001a070 con 0x7f088c0d0f90 2024-03-29T04:07:38.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.125+0000 7f087affd640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f086c06c530 0x7f086c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.125+0000 7f087affd640 1 -- 172.21.15.97:0/3568864580 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f088008b190 con 0x7f088c0d0f90 2024-03-29T04:07:38.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.125+0000 7f088a59c640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f086c06c530 0x7f086c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.125+0000 7f088a59c640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f086c06c530 0x7f086c06e9f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f087c036000 tx=0x7f087c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:38.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.129+0000 7f087affd640 1 -- 172.21.15.97:0/3568864580 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f08800594b0 con 0x7f088c0d0f90 2024-03-29T04:07:38.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.308+0000 7f0890bf4640 1 -- 172.21.15.97:0/3568864580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f088c0d1c50 con 0x7f088c0d0f90 2024-03-29T04:07:38.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.308+0000 7f087affd640 1 -- 172.21.15.97:0/3568864580 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f088005d7b0 con 0x7f088c0d0f90 2024-03-29T04:07:38.288 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:38.289 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-03-29T04:07:38.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:07:38.289 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-03-29T04:07:38.289 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-03-29T04:07:38.289 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:07:38.289 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-03-29T04:07:38.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.310+0000 7f0890bf4640 1 -- 172.21.15.97:0/3568864580 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f086c06c530 msgr2=0x7f086c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:38.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.310+0000 7f0890bf4640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f086c06c530 0x7f086c06e9f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f087c036000 tx=0x7f087c030000 comp rx=0 tx=0).stop 2024-03-29T04:07:38.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.310+0000 7f0890bf4640 1 -- 172.21.15.97:0/3568864580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c0d0f90 msgr2=0x7f088c15ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:38.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.310+0000 7f0890bf4640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c0d0f90 0x7f088c15ab80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f088000e870 tx=0x7f088000ed30 comp rx=0 tx=0).stop 2024-03-29T04:07:38.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.310+0000 7f0890bf4640 1 -- 172.21.15.97:0/3568864580 shutdown_connections 2024-03-29T04:07:38.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.310+0000 7f0890bf4640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f086c06c530 0x7f086c06e9f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.310+0000 7f0890bf4640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f088c15e920 0x7f088c1f8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.310+0000 7f0890bf4640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c0d0f90 0x7f088c15ab80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.310+0000 7f0890bf4640 1 --2- 172.21.15.97:0/3568864580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f088c0cfde0 0x7f088c158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.310+0000 7f0890bf4640 1 -- 172.21.15.97:0/3568864580 >> 172.21.15.97:0/3568864580 conn(0x7f088c08bfc0 msgr2=0x7f088c06dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:38.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.310+0000 7f0890bf4640 1 -- 172.21.15.97:0/3568864580 shutdown_connections 2024-03-29T04:07:38.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.310+0000 7f0890bf4640 1 -- 172.21.15.97:0/3568864580 wait complete. 2024-03-29T04:07:38.301 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T04:07:38.301 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T04:07:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.426+0000 7f4e68f60640 1 Processor -- start 2024-03-29T04:07:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.427+0000 7f4e68f60640 1 -- start start 2024-03-29T04:07:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.427+0000 7f4e68f60640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e6415e8d0 0x7f4e640b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.427+0000 7f4e68f60640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4e6415ecb0 0x7f4e640d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.427+0000 7f4e68f60640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e640cf260 con 0x7f4e640d2990 2024-03-29T04:07:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.427+0000 7f4e68f60640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f4e640b4ac0 con 0x7f4e6415ecb0 2024-03-29T04:07:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.427+0000 7f4e68f60640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4e640b4d90 con 0x7f4e6415e8d0 2024-03-29T04:07:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.427+0000 7f4e62d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4e640d2990 0x7f4e640b5c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52642/0 (socket says 172.21.15.97:52642) 2024-03-29T04:07:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.427+0000 7f4e62d9d640 1 -- 172.21.15.97:0/2683865112 learned_addr learned my addr 172.21.15.97:0/2683865112 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e59d9b640 1 --2- 172.21.15.97:0/2683865112 >> v2:172.21.15.154:3300/0 conn(0x7f4e6415ecb0 0x7f4e640d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e6259c640 1 --2- 172.21.15.97:0/2683865112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e6415e8d0 0x7f4e640b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 263580174 0 0) 0x7f4e640cf260 con 0x7f4e640d2990 2024-03-29T04:07:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e50003520 con 0x7f4e640d2990 2024-03-29T04:07:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1361978781 0 0) 0x7f4e50003520 con 0x7f4e640d2990 2024-03-29T04:07:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e6415e8d0 msgr2=0x7f4e640b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 --2- 172.21.15.97:0/2683865112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e6415e8d0 0x7f4e640b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 >> v2:172.21.15.154:3300/0 conn(0x7f4e6415ecb0 msgr2=0x7f4e640d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 --2- 172.21.15.97:0/2683865112 >> v2:172.21.15.154:3300/0 conn(0x7f4e6415ecb0 0x7f4e640d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e640b5070 con 0x7f4e640d2990 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f4e5c001a10 con 0x7f4e640d2990 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 >> v1:172.21.15.97:6789/0 conn(0x7f4e640d2990 legacy=0x7f4e640b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 --2- 172.21.15.97:0/2683865112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e500039b0 0x7f4e50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 --2- 172.21.15.97:0/2683865112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e50004800 0x7f4e50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 --2- 172.21.15.97:0/2683865112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e6415e8d0 0x7f4e500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4e640b4ac0 con 0x7f4e50004800 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4e640b4d90 con 0x7f4e500039b0 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4e5000b9c0 con 0x7f4e6415e8d0 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e59d9b640 1 --2- 172.21.15.97:0/2683865112 >> v2:172.21.15.154:3300/0 conn(0x7f4e6415ecb0 0x7f4e640d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e59d9b640 1 --2- 172.21.15.97:0/2683865112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e50004800 0x7f4e50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e6259c640 1 --2- 172.21.15.97:0/2683865112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e500039b0 0x7f4e50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e62d9d640 1 --2- 172.21.15.97:0/2683865112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e6415e8d0 0x7f4e500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e59d9b640 1 -- 172.21.15.97:0/2683865112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e6415e8d0 msgr2=0x7f4e500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e59d9b640 1 --2- 172.21.15.97:0/2683865112 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e6415e8d0 0x7f4e500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e59d9b640 1 -- 172.21.15.97:0/2683865112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e500039b0 msgr2=0x7f4e50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e59d9b640 1 --2- 172.21.15.97:0/2683865112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e500039b0 0x7f4e50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.428+0000 7f4e59d9b640 1 -- 172.21.15.97:0/2683865112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e50003520 con 0x7f4e50004800 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.429+0000 7f4e59d9b640 1 --2- 172.21.15.97:0/2683865112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e50004800 0x7f4e50004c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009c60 tx=0x7f4e50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cd7fa92bbcc607 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.429+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c009830 con 0x7f4e50004800 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.429+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e4c004020 con 0x7f4e50004800 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.429+0000 7f4e61d9b640 1 -- 172.21.15.97:0/2683865112 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c004410 con 0x7f4e50004800 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.430+0000 7f4e68f60640 1 -- 172.21.15.97:0/2683865112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e50004800 msgr2=0x7f4e50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.430+0000 7f4e68f60640 1 --2- 172.21.15.97:0/2683865112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e50004800 0x7f4e50004c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009c60 tx=0x7f4e50005830 comp rx=0 tx=0).stop 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.430+0000 7f4e62d9d640 1 -- 172.21.15.97:0/2683865112 reap_dead start 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.430+0000 7f4e68f60640 1 -- 172.21.15.97:0/2683865112 shutdown_connections 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.430+0000 7f4e68f60640 1 -- 172.21.15.97:0/2683865112 >> 172.21.15.97:0/2683865112 conn(0x7f4e6409d940 msgr2=0x7f4e6408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.430+0000 7f4e68f60640 1 -- 172.21.15.97:0/2683865112 shutdown_connections 2024-03-29T04:07:38.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.430+0000 7f4e68f60640 1 -- 172.21.15.97:0/2683865112 wait complete. 2024-03-29T04:07:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.431+0000 7f4e68f60640 1 Processor -- start 2024-03-29T04:07:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.431+0000 7f4e68f60640 1 -- start start 2024-03-29T04:07:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e68f60640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e50004800 0x7f4e641531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e68f60640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e640cd5c0 0x7f4e640cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e68f60640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e640cc900 0x7f4e640ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e68f60640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4e640d0750 con 0x7f4e640cc900 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e68f60640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4e640b5630 con 0x7f4e640cd5c0 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e68f60640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4e640d08d0 con 0x7f4e50004800 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e62d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e640cc900 0x7f4e640ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e62d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e640cc900 0x7f4e640ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60250/0 (socket says 172.21.15.97:60250) 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e62d9d640 1 -- 172.21.15.97:0/2384863728 learned_addr learned my addr 172.21.15.97:0/2384863728 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e6259c640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e50004800 0x7f4e641531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e59d9b640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e640cd5c0 0x7f4e640cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e62d9d640 1 -- 172.21.15.97:0/2384863728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e50004800 msgr2=0x7f4e641531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e62d9d640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e50004800 0x7f4e641531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e62d9d640 1 -- 172.21.15.97:0/2384863728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e640cd5c0 msgr2=0x7f4e640cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e62d9d640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e640cd5c0 0x7f4e640cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e62d9d640 1 -- 172.21.15.97:0/2384863728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e641f7dd0 con 0x7f4e640cc900 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.432+0000 7f4e62d9d640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e640cc900 0x7f4e640ccd60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e640b44d0 tx=0x7f4e5c007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.433+0000 7f4e5b7fe640 1 -- 172.21.15.97:0/2384863728 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00abe0 con 0x7f4e640cc900 2024-03-29T04:07:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.433+0000 7f4e5b7fe640 1 -- 172.21.15.97:0/2384863728 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5c018070 con 0x7f4e640cc900 2024-03-29T04:07:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.433+0000 7f4e68f60640 1 -- 172.21.15.97:0/2384863728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e641f80b0 con 0x7f4e640cc900 2024-03-29T04:07:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.433+0000 7f4e5b7fe640 1 -- 172.21.15.97:0/2384863728 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0133f0 con 0x7f4e640cc900 2024-03-29T04:07:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.433+0000 7f4e68f60640 1 -- 172.21.15.97:0/2384863728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e641f8670 con 0x7f4e640cc900 2024-03-29T04:07:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.433+0000 7f4e68f60640 1 -- 172.21.15.97:0/2384863728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e64068640 con 0x7f4e640cc900 2024-03-29T04:07:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.434+0000 7f4e5b7fe640 1 -- 172.21.15.97:0/2384863728 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4e5c013590 con 0x7f4e640cc900 2024-03-29T04:07:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.434+0000 7f4e5b7fe640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4e4406c530 0x7f4e4406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.434+0000 7f4e5b7fe640 1 -- 172.21.15.97:0/2384863728 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f4e5c08b9e0 con 0x7f4e640cc900 2024-03-29T04:07:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.434+0000 7f4e6259c640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4e4406c530 0x7f4e4406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.435+0000 7f4e6259c640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4e4406c530 0x7f4e4406e9f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e540098a0 tx=0x7f4e54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:38.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.439+0000 7f4e5b7fe640 1 -- 172.21.15.97:0/2384863728 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4e5c05a350 con 0x7f4e640cc900 2024-03-29T04:07:38.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:38.614+0000 7f4e68f60640 1 -- 172.21.15.97:0/2384863728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4e640d32a0 con 0x7f4e640cc900 2024-03-29T04:07:39.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.402+0000 7f4e5b7fe640 1 -- 172.21.15.97:0/2384863728 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f4e5c059c50 con 0x7f4e640cc900 2024-03-29T04:07:39.382 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-03-29T04:07:39.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.405+0000 7f4e68f60640 1 -- 172.21.15.97:0/2384863728 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4e4406c530 msgr2=0x7f4e4406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:39.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.405+0000 7f4e68f60640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4e4406c530 0x7f4e4406e9f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e540098a0 tx=0x7f4e54008040 comp rx=0 tx=0).stop 2024-03-29T04:07:39.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.405+0000 7f4e68f60640 1 -- 172.21.15.97:0/2384863728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e640cc900 msgr2=0x7f4e640ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:39.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.405+0000 7f4e68f60640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e640cc900 0x7f4e640ccd60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e640b44d0 tx=0x7f4e5c007b20 comp rx=0 tx=0).stop 2024-03-29T04:07:39.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.405+0000 7f4e68f60640 1 -- 172.21.15.97:0/2384863728 shutdown_connections 2024-03-29T04:07:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.405+0000 7f4e68f60640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4e4406c530 0x7f4e4406e9f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.405+0000 7f4e68f60640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e640cc900 0x7f4e640ccd60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.405+0000 7f4e68f60640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e640cd5c0 0x7f4e640cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.405+0000 7f4e68f60640 1 --2- 172.21.15.97:0/2384863728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e50004800 0x7f4e641531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.405+0000 7f4e68f60640 1 -- 172.21.15.97:0/2384863728 >> 172.21.15.97:0/2384863728 conn(0x7f4e6409d940 msgr2=0x7f4e640b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.405+0000 7f4e68f60640 1 -- 172.21.15.97:0/2384863728 shutdown_connections 2024-03-29T04:07:39.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.405+0000 7f4e68f60640 1 -- 172.21.15.97:0/2384863728 wait complete. 2024-03-29T04:07:39.401 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T04:07:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.532+0000 7f8622655640 1 Processor -- start 2024-03-29T04:07:39.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.532+0000 7f8622655640 1 -- start start 2024-03-29T04:07:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f8622655640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f861c0d3430 0x7f861c0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f8622655640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f861c0d2390 con 0x7f861c0d1010 2024-03-29T04:07:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f8622655640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f861c0d2670 con 0x7f861c0d0210 2024-03-29T04:07:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f8622655640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f861c0d5060 con 0x7f861c0d3430 2024-03-29T04:07:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f861bfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f861c0d1010 0x7f861c0d13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:52646/0 (socket says 172.21.15.97:52646) 2024-03-29T04:07:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f861bfff640 1 -- 172.21.15.97:0/2661219598 learned_addr learned my addr 172.21.15.97:0/2661219598 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f861b7fe640 1 --2- 172.21.15.97:0/2661219598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f861c0d3430 0x7f861c0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4127448624 0 0) 0x7f861c0d2390 con 0x7f861c0d1010 2024-03-29T04:07:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8604003520 con 0x7f861c0d1010 2024-03-29T04:07:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1804160254 0 0) 0x7f8604003520 con 0x7f861c0d1010 2024-03-29T04:07:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f861c0d3430 msgr2=0x7f861c0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:39.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f861affd640 1 --2- 172.21.15.97:0/2661219598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f861c0d3430 0x7f861c0cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 >> v1:172.21.15.154:6789/0 conn(0x7f861c0d0210 legacy=0x7f861c0b7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c0d2950 con 0x7f861c0d1010 2024-03-29T04:07:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.533+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f860c001670 con 0x7f861c0d1010 2024-03-29T04:07:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 >> v1:172.21.15.97:6789/0 conn(0x7f861c0d1010 legacy=0x7f861c0d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861affd640 1 --2- 172.21.15.97:0/2661219598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86040039e0 0x7f8604003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861affd640 1 --2- 172.21.15.97:0/2661219598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f861c0d3430 0x7f8604004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861affd640 1 --2- 172.21.15.97:0/2661219598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8604004d40 0x7f86040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f861c0d5060 con 0x7f86040039e0 2024-03-29T04:07:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f860400b9c0 con 0x7f8604004d40 2024-03-29T04:07:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f860400bbc0 con 0x7f861c0d3430 2024-03-29T04:07:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861bfff640 1 --2- 172.21.15.97:0/2661219598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86040039e0 0x7f8604003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861b7fe640 1 --2- 172.21.15.97:0/2661219598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f861c0d3430 0x7f8604004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:39.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f8620bf2640 1 --2- 172.21.15.97:0/2661219598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8604004d40 0x7f86040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861bfff640 1 -- 172.21.15.97:0/2661219598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f861c0d3430 msgr2=0x7f8604004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861bfff640 1 --2- 172.21.15.97:0/2661219598 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f861c0d3430 0x7f8604004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861bfff640 1 -- 172.21.15.97:0/2661219598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8604004d40 msgr2=0x7f86040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861bfff640 1 --2- 172.21.15.97:0/2661219598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8604004d40 0x7f86040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.534+0000 7f861bfff640 1 -- 172.21.15.97:0/2661219598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8604003520 con 0x7f86040039e0 2024-03-29T04:07:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.535+0000 7f861bfff640 1 --2- 172.21.15.97:0/2661219598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86040039e0 0x7f8604003dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f860c00c190 tx=0x7f8604005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368d3e408bafbb66 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.535+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c007cc0 con 0x7f86040039e0 2024-03-29T04:07:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.535+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860c030070 con 0x7f86040039e0 2024-03-29T04:07:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.535+0000 7f861affd640 1 -- 172.21.15.97:0/2661219598 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c005d10 con 0x7f86040039e0 2024-03-29T04:07:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.535+0000 7f8622655640 1 -- 172.21.15.97:0/2661219598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86040039e0 msgr2=0x7f8604003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.535+0000 7f8622655640 1 --2- 172.21.15.97:0/2661219598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86040039e0 0x7f8604003dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f860c00c190 tx=0x7f8604005830 comp rx=0 tx=0).stop 2024-03-29T04:07:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.535+0000 7f8620bf2640 1 -- 172.21.15.97:0/2661219598 reap_dead start 2024-03-29T04:07:39.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.535+0000 7f8622655640 1 -- 172.21.15.97:0/2661219598 shutdown_connections 2024-03-29T04:07:39.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.535+0000 7f8622655640 1 -- 172.21.15.97:0/2661219598 >> 172.21.15.97:0/2661219598 conn(0x7f861c08bfc0 msgr2=0x7f861c0d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:39.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.535+0000 7f8622655640 1 -- 172.21.15.97:0/2661219598 shutdown_connections 2024-03-29T04:07:39.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.535+0000 7f8622655640 1 -- 172.21.15.97:0/2661219598 wait complete. 2024-03-29T04:07:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.536+0000 7f8622655640 1 Processor -- start 2024-03-29T04:07:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.537+0000 7f8622655640 1 -- start start 2024-03-29T04:07:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.537+0000 7f8622655640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f86040039e0 0x7f861c1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.537+0000 7f8622655640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f861c0b9550 0x7f861c1e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.537+0000 7f8622655640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f861c1ea2f0 0x7f861c1f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.537+0000 7f8622655640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f861c15eaf0 con 0x7f861c0b9550 2024-03-29T04:07:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.537+0000 7f8622655640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f861c0d2f10 con 0x7f861c1ea2f0 2024-03-29T04:07:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.537+0000 7f8622655640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f861c15ec70 con 0x7f86040039e0 2024-03-29T04:07:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.537+0000 7f861b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f861c0b9550 0x7f861c1e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.537+0000 7f861b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f861c0b9550 0x7f861c1e9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60270/0 (socket says 172.21.15.97:60270) 2024-03-29T04:07:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.537+0000 7f861b7fe640 1 -- 172.21.15.97:0/2721540731 learned_addr learned my addr 172.21.15.97:0/2721540731 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f8620bf2640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f861c1ea2f0 0x7f861c1f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f861bfff640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f86040039e0 0x7f861c1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f861b7fe640 1 -- 172.21.15.97:0/2721540731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f86040039e0 msgr2=0x7f861c1590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f861b7fe640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f86040039e0 0x7f861c1590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f861b7fe640 1 -- 172.21.15.97:0/2721540731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f861c1ea2f0 msgr2=0x7f861c1f7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f861b7fe640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f861c1ea2f0 0x7f861c1f7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f861b7fe640 1 -- 172.21.15.97:0/2721540731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c1e8d50 con 0x7f861c0b9550 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f8620bf2640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f861c1ea2f0 0x7f861c1f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f861bfff640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f86040039e0 0x7f861c1590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f861b7fe640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f861c0b9550 0x7f861c1e9db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f86100119d0 tx=0x7f8610011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f8618ff9640 1 -- 172.21.15.97:0/2721540731 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861000fb50 con 0x7f861c0b9550 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f8618ff9640 1 -- 172.21.15.97:0/2721540731 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861000fcf0 con 0x7f861c0b9550 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f8622655640 1 -- 172.21.15.97:0/2721540731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861c1e9030 con 0x7f861c0b9550 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f8618ff9640 1 -- 172.21.15.97:0/2721540731 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861000fea0 con 0x7f861c0b9550 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.538+0000 7f8622655640 1 -- 172.21.15.97:0/2721540731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f861c0b76a0 con 0x7f861c0b9550 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.539+0000 7f8622655640 1 -- 172.21.15.97:0/2721540731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f861c0a2570 con 0x7f861c0b9550 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.539+0000 7f8618ff9640 1 -- 172.21.15.97:0/2721540731 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8610012040 con 0x7f861c0b9550 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.540+0000 7f8618ff9640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f860806c530 0x7f860806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:39.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.540+0000 7f861bfff640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f860806c530 0x7f860806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:39.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.540+0000 7f8618ff9640 1 -- 172.21.15.97:0/2721540731 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f861008ba70 con 0x7f861c0b9550 2024-03-29T04:07:39.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.540+0000 7f861bfff640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f860806c530 0x7f860806e9f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f860c034000 tx=0x7f860c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:39.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.544+0000 7f8618ff9640 1 -- 172.21.15.97:0/2721540731 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8610019030 con 0x7f861c0b9550 2024-03-29T04:07:39.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:39.720+0000 7f8622655640 1 -- 172.21.15.97:0/2721540731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f861c1ea6d0 con 0x7f861c0b9550 2024-03-29T04:07:40.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:07:40.158 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:07:40.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.407+0000 7f8618ff9640 1 -- 172.21.15.97:0/2721540731 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f861005a300 con 0x7f861c0b9550 2024-03-29T04:07:40.387 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-03-29T04:07:40.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.409+0000 7f8622655640 1 -- 172.21.15.97:0/2721540731 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f860806c530 msgr2=0x7f860806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:40.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.409+0000 7f8622655640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f860806c530 0x7f860806e9f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f860c034000 tx=0x7f860c009030 comp rx=0 tx=0).stop 2024-03-29T04:07:40.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.409+0000 7f8622655640 1 -- 172.21.15.97:0/2721540731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f861c0b9550 msgr2=0x7f861c1e9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.409+0000 7f8622655640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f861c0b9550 0x7f861c1e9db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f86100119d0 tx=0x7f8610011e90 comp rx=0 tx=0).stop 2024-03-29T04:07:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.409+0000 7f8622655640 1 -- 172.21.15.97:0/2721540731 shutdown_connections 2024-03-29T04:07:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.409+0000 7f8622655640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f860806c530 0x7f860806e9f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.409+0000 7f8622655640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f861c1ea2f0 0x7f861c1f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.409+0000 7f8622655640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f861c0b9550 0x7f861c1e9db0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.409+0000 7f8622655640 1 --2- 172.21.15.97:0/2721540731 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f86040039e0 0x7f861c1590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.409+0000 7f8622655640 1 -- 172.21.15.97:0/2721540731 >> 172.21.15.97:0/2721540731 conn(0x7f861c08bfc0 msgr2=0x7f861c15edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.410+0000 7f8622655640 1 -- 172.21.15.97:0/2721540731 shutdown_connections 2024-03-29T04:07:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.410+0000 7f8622655640 1 -- 172.21.15.97:0/2721540731 wait complete. 2024-03-29T04:07:40.409 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T04:07:40.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.533+0000 7f88c233d640 1 Processor -- start 2024-03-29T04:07:40.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.533+0000 7f88c233d640 1 -- start start 2024-03-29T04:07:40.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.533+0000 7f88c233d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88bc0cfde0 0x7f88bc0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:40.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.533+0000 7f88c233d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f88bc0d0f90 0x7f88bc0d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:40.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.533+0000 7f88c233d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88bc15ed00 con 0x7f88bc15e920 2024-03-29T04:07:40.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.533+0000 7f88c233d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f88bc0d21e0 con 0x7f88bc0d0f90 2024-03-29T04:07:40.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.533+0000 7f88c233d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f88bc0d2480 con 0x7f88bc0cfde0 2024-03-29T04:07:40.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88bbfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f88bc15e920 0x7f88bc0d3240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50266/0 (socket says 172.21.15.97:50266) 2024-03-29T04:07:40.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88bbfff640 1 -- 172.21.15.97:0/3551651621 learned_addr learned my addr 172.21.15.97:0/3551651621 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:40.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88c08da640 1 --2- 172.21.15.97:0/3551651621 >> v2:172.21.15.154:3300/0 conn(0x7f88bc0d0f90 0x7f88bc0d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:40.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 856193666 0 0) 0x7f88bc15ed00 con 0x7f88bc15e920 2024-03-29T04:07:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88bb7fe640 1 --2- 172.21.15.97:0/3551651621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88bc0cfde0 0x7f88bc0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88a4003520 con 0x7f88bc15e920 2024-03-29T04:07:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 217640269 0 0) 0x7f88a4003520 con 0x7f88bc15e920 2024-03-29T04:07:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88bc0cfde0 msgr2=0x7f88bc0d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 --2- 172.21.15.97:0/3551651621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88bc0cfde0 0x7f88bc0d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 >> v2:172.21.15.154:3300/0 conn(0x7f88bc0d0f90 msgr2=0x7f88bc0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 --2- 172.21.15.97:0/3551651621 >> v2:172.21.15.154:3300/0 conn(0x7f88bc0d0f90 0x7f88bc0d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc0d2760 con 0x7f88bc15e920 2024-03-29T04:07:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f88ac001670 con 0x7f88bc15e920 2024-03-29T04:07:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 >> v1:172.21.15.97:6789/0 conn(0x7f88bc15e920 legacy=0x7f88bc0d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 --2- 172.21.15.97:0/3551651621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88a40039b0 0x7f88a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:40.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 --2- 172.21.15.97:0/3551651621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88a4004800 0x7f88a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 --2- 172.21.15.97:0/3551651621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88bc0cfde0 0x7f88a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc0d21e0 con 0x7f88a40039b0 2024-03-29T04:07:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc0d2480 con 0x7f88a4004800 2024-03-29T04:07:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f88a400b9c0 con 0x7f88bc0cfde0 2024-03-29T04:07:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88bbfff640 1 --2- 172.21.15.97:0/3551651621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88a40039b0 0x7f88a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88bb7fe640 1 --2- 172.21.15.97:0/3551651621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88a4004800 0x7f88a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88c08da640 1 --2- 172.21.15.97:0/3551651621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88bc0cfde0 0x7f88a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88bbfff640 1 -- 172.21.15.97:0/3551651621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88bc0cfde0 msgr2=0x7f88a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88bbfff640 1 --2- 172.21.15.97:0/3551651621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88bc0cfde0 0x7f88a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88bbfff640 1 -- 172.21.15.97:0/3551651621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88a4004800 msgr2=0x7f88a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88bbfff640 1 --2- 172.21.15.97:0/3551651621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88a4004800 0x7f88a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.534+0000 7f88bbfff640 1 -- 172.21.15.97:0/3551651621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a4003520 con 0x7f88a40039b0 2024-03-29T04:07:40.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.535+0000 7f88bbfff640 1 --2- 172.21.15.97:0/3551651621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88a40039b0 0x7f88a4003dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00c210 tx=0x7f88a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d560f4cc4b69c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:40.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.535+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac007d30 con 0x7f88a40039b0 2024-03-29T04:07:40.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.535+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ac030070 con 0x7f88a40039b0 2024-03-29T04:07:40.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.535+0000 7f88baffd640 1 -- 172.21.15.97:0/3551651621 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac02cd60 con 0x7f88a40039b0 2024-03-29T04:07:40.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.536+0000 7f88c233d640 1 -- 172.21.15.97:0/3551651621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88a40039b0 msgr2=0x7f88a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:40.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.536+0000 7f88c233d640 1 --2- 172.21.15.97:0/3551651621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88a40039b0 0x7f88a4003dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00c210 tx=0x7f88a4005830 comp rx=0 tx=0).stop 2024-03-29T04:07:40.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.536+0000 7f88c08da640 1 -- 172.21.15.97:0/3551651621 reap_dead start 2024-03-29T04:07:40.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.536+0000 7f88c233d640 1 -- 172.21.15.97:0/3551651621 shutdown_connections 2024-03-29T04:07:40.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.536+0000 7f88c233d640 1 -- 172.21.15.97:0/3551651621 >> 172.21.15.97:0/3551651621 conn(0x7f88bc08bfc0 msgr2=0x7f88bc0d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:40.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.536+0000 7f88c233d640 1 -- 172.21.15.97:0/3551651621 shutdown_connections 2024-03-29T04:07:40.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.536+0000 7f88c233d640 1 -- 172.21.15.97:0/3551651621 wait complete. 2024-03-29T04:07:40.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.537+0000 7f88c233d640 1 Processor -- start 2024-03-29T04:07:40.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.537+0000 7f88c233d640 1 -- start start 2024-03-29T04:07:40.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88c233d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88a40039b0 0x7f88bc159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:40.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88c233d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88bc0b4070 0x7f88bc160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:40.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88c233d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88bc0b44f0 0x7f88bc160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:40.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88c233d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc0d4ef0 con 0x7f88bc0b4070 2024-03-29T04:07:40.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88c233d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc0d2d20 con 0x7f88bc0b44f0 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88c233d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f88bc0d5070 con 0x7f88a40039b0 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88bb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88bc0b4070 0x7f88bc160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88bb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88bc0b4070 0x7f88bc160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38598/0 (socket says 172.21.15.97:38598) 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88bb7fe640 1 -- 172.21.15.97:0/736486344 learned_addr learned my addr 172.21.15.97:0/736486344 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88bbfff640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88a40039b0 0x7f88bc159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88c08da640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88bc0b44f0 0x7f88bc160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88bb7fe640 1 -- 172.21.15.97:0/736486344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88a40039b0 msgr2=0x7f88bc159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88bb7fe640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88a40039b0 0x7f88bc159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88bb7fe640 1 -- 172.21.15.97:0/736486344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88bc0b44f0 msgr2=0x7f88bc160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88bb7fe640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88bc0b44f0 0x7f88bc160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88bb7fe640 1 -- 172.21.15.97:0/736486344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc0ccda0 con 0x7f88bc0b4070 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88c08da640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88bc0b44f0 0x7f88bc160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.538+0000 7f88bb7fe640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88bc0b4070 0x7f88bc160330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88b00118e0 tx=0x7f88b0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.539+0000 7f88b8ff9640 1 -- 172.21.15.97:0/736486344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b000faf0 con 0x7f88bc0b4070 2024-03-29T04:07:40.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.539+0000 7f88b8ff9640 1 -- 172.21.15.97:0/736486344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b000fc90 con 0x7f88bc0b4070 2024-03-29T04:07:40.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.539+0000 7f88b8ff9640 1 -- 172.21.15.97:0/736486344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0014440 con 0x7f88bc0b4070 2024-03-29T04:07:40.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.539+0000 7f88c233d640 1 -- 172.21.15.97:0/736486344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88bc0b4f90 con 0x7f88bc0b4070 2024-03-29T04:07:40.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.539+0000 7f88c233d640 1 -- 172.21.15.97:0/736486344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88bc0b5550 con 0x7f88bc0b4070 2024-03-29T04:07:40.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.540+0000 7f88c233d640 1 -- 172.21.15.97:0/736486344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88bc0a2560 con 0x7f88bc0b4070 2024-03-29T04:07:40.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.540+0000 7f88b8ff9640 1 -- 172.21.15.97:0/736486344 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f88b0012040 con 0x7f88bc0b4070 2024-03-29T04:07:40.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.541+0000 7f88b8ff9640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f88a806c560 0x7f88a806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:40.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.541+0000 7f88bbfff640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f88a806c560 0x7f88a806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:40.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.541+0000 7f88b8ff9640 1 -- 172.21.15.97:0/736486344 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f88b008bbe0 con 0x7f88bc0b4070 2024-03-29T04:07:40.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.541+0000 7f88bbfff640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f88a806c560 0x7f88a806ea20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f88ac034000 tx=0x7f88ac009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:40.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.545+0000 7f88b8ff9640 1 -- 172.21.15.97:0/736486344 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f88bc0ccda0 con 0x7f88bc0b4070 2024-03-29T04:07:40.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:40.759+0000 7f88c233d640 1 -- 172.21.15.97:0/736486344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f88bc04f9d0 con 0x7f88bc0b4070 2024-03-29T04:07:41.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.421+0000 7f88b8ff9640 1 -- 172.21.15.97:0/736486344 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f88b0019030 con 0x7f88bc0b4070 2024-03-29T04:07:41.403 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 2 and data pool 3 2024-03-29T04:07:41.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.429+0000 7f88c233d640 1 -- 172.21.15.97:0/736486344 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f88a806c560 msgr2=0x7f88a806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.429+0000 7f88c233d640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f88a806c560 0x7f88a806ea20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f88ac034000 tx=0x7f88ac009030 comp rx=0 tx=0).stop 2024-03-29T04:07:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.429+0000 7f88c233d640 1 -- 172.21.15.97:0/736486344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88bc0b4070 msgr2=0x7f88bc160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.429+0000 7f88c233d640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88bc0b4070 0x7f88bc160330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88b00118e0 tx=0x7f88b0011da0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.430+0000 7f88c233d640 1 -- 172.21.15.97:0/736486344 shutdown_connections 2024-03-29T04:07:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.430+0000 7f88c233d640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f88a806c560 0x7f88a806ea20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.430+0000 7f88c233d640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88bc0b44f0 0x7f88bc160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.430+0000 7f88c233d640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88bc0b4070 0x7f88bc160330 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.430+0000 7f88c233d640 1 --2- 172.21.15.97:0/736486344 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88a40039b0 0x7f88bc159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.430+0000 7f88c233d640 1 -- 172.21.15.97:0/736486344 >> 172.21.15.97:0/736486344 conn(0x7f88bc08bfc0 msgr2=0x7f88bc0b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.430+0000 7f88c233d640 1 -- 172.21.15.97:0/736486344 shutdown_connections 2024-03-29T04:07:41.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.430+0000 7f88c233d640 1 -- 172.21.15.97:0/736486344 wait complete. 2024-03-29T04:07:41.425 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:41.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.545+0000 7f2a42560640 1 Processor -- start 2024-03-29T04:07:41.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.546+0000 7f2a42560640 1 -- start start 2024-03-29T04:07:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.546+0000 7f2a42560640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a3c06f3f0 0x7f2a3c072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.546+0000 7f2a42560640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2a3c0730c0 0x7f2a3c15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.546+0000 7f2a42560640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c071ae0 con 0x7f2a3c15b370 2024-03-29T04:07:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.546+0000 7f2a42560640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2a3c0b39d0 con 0x7f2a3c06f3f0 2024-03-29T04:07:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.546+0000 7f2a42560640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f2a3c071dc0 con 0x7f2a3c0730c0 2024-03-29T04:07:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.546+0000 7f2a3b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a3c06f3f0 0x7f2a3c072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.546+0000 7f2a3bfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2a3c15b370 0x7f2a3c0b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50282/0 (socket says 172.21.15.97:50282) 2024-03-29T04:07:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.546+0000 7f2a3b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a3c06f3f0 0x7f2a3c072b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38612/0 (socket says 172.21.15.97:38612) 2024-03-29T04:07:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.546+0000 7f2a3bfff640 1 -- 172.21.15.97:0/2183889152 learned_addr learned my addr 172.21.15.97:0/2183889152 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.547+0000 7f2a40afd640 1 --2- 172.21.15.97:0/2183889152 >> v2:172.21.15.154:3300/0 conn(0x7f2a3c0730c0 0x7f2a3c15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:41.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.547+0000 7f2a3b7fe640 1 -- 172.21.15.97:0/2183889152 >> v2:172.21.15.154:3300/0 conn(0x7f2a3c0730c0 msgr2=0x7f2a3c15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:41.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.547+0000 7f2a3b7fe640 1 --2- 172.21.15.97:0/2183889152 >> v2:172.21.15.154:3300/0 conn(0x7f2a3c0730c0 0x7f2a3c15f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.547+0000 7f2a3b7fe640 1 -- 172.21.15.97:0/2183889152 >> v1:172.21.15.97:6789/0 conn(0x7f2a3c15b370 legacy=0x7f2a3c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:41.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.547+0000 7f2a3affd640 1 -- 172.21.15.97:0/2183889152 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3161014665 0 0) 0x7f2a3c071ae0 con 0x7f2a3c15b370 2024-03-29T04:07:41.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.547+0000 7f2a3b7fe640 1 -- 172.21.15.97:0/2183889152 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c0720a0 con 0x7f2a3c06f3f0 2024-03-29T04:07:41.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.547+0000 7f2a3b7fe640 1 --2- 172.21.15.97:0/2183889152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a3c06f3f0 0x7f2a3c072b80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c06f850 tx=0x7f2a3002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2952d1c153471279 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:41.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.548+0000 7f2a3affd640 1 -- 172.21.15.97:0/2183889152 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30035020 con 0x7f2a3c06f3f0 2024-03-29T04:07:41.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.548+0000 7f2a3affd640 1 -- 172.21.15.97:0/2183889152 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a30031830 con 0x7f2a3c06f3f0 2024-03-29T04:07:41.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.548+0000 7f2a3affd640 1 -- 172.21.15.97:0/2183889152 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30031c00 con 0x7f2a3c06f3f0 2024-03-29T04:07:41.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.548+0000 7f2a42560640 1 -- 172.21.15.97:0/2183889152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a3c06f3f0 msgr2=0x7f2a3c072b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:41.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.548+0000 7f2a42560640 1 --2- 172.21.15.97:0/2183889152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a3c06f3f0 0x7f2a3c072b80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c06f850 tx=0x7f2a3002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.548+0000 7f2a42560640 1 -- 172.21.15.97:0/2183889152 shutdown_connections 2024-03-29T04:07:41.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.548+0000 7f2a42560640 1 --2- 172.21.15.97:0/2183889152 >> v2:172.21.15.154:3300/0 conn(0x7f2a3c0730c0 0x7f2a3c15f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.548+0000 7f2a42560640 1 --2- 172.21.15.97:0/2183889152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a3c06f3f0 0x7f2a3c072b80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.548+0000 7f2a42560640 1 -- 172.21.15.97:0/2183889152 >> 172.21.15.97:0/2183889152 conn(0x7f2a3c06b510 msgr2=0x7f2a3c06b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:41.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.548+0000 7f2a42560640 1 -- 172.21.15.97:0/2183889152 shutdown_connections 2024-03-29T04:07:41.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.548+0000 7f2a42560640 1 -- 172.21.15.97:0/2183889152 wait complete. 2024-03-29T04:07:41.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.549+0000 7f2a42560640 1 Processor -- start 2024-03-29T04:07:41.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.550+0000 7f2a42560640 1 -- start start 2024-03-29T04:07:41.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.550+0000 7f2a42560640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a3c06f3f0 0x7f2a3c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:41.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.550+0000 7f2a42560640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a3c0730c0 0x7f2a3c14ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:41.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.550+0000 7f2a42560640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a3c15b370 0x7f2a3c15e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:41.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.550+0000 7f2a42560640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c073550 con 0x7f2a3c0730c0 2024-03-29T04:07:41.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.550+0000 7f2a42560640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c072660 con 0x7f2a3c15b370 2024-03-29T04:07:41.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.550+0000 7f2a42560640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2a3c15b750 con 0x7f2a3c06f3f0 2024-03-29T04:07:41.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.550+0000 7f2a3b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a3c0730c0 0x7f2a3c14ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:41.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.550+0000 7f2a3b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a3c0730c0 0x7f2a3c14ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38620/0 (socket says 172.21.15.97:38620) 2024-03-29T04:07:41.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.550+0000 7f2a3b7fe640 1 -- 172.21.15.97:0/1673200443 learned_addr learned my addr 172.21.15.97:0/1673200443 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:41.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.550+0000 7f2a3bfff640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a3c06f3f0 0x7f2a3c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:41.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.550+0000 7f2a40afd640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a3c15b370 0x7f2a3c15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:41.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.551+0000 7f2a3b7fe640 1 -- 172.21.15.97:0/1673200443 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a3c06f3f0 msgr2=0x7f2a3c159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.551+0000 7f2a3b7fe640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a3c06f3f0 0x7f2a3c159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.551+0000 7f2a3b7fe640 1 -- 172.21.15.97:0/1673200443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a3c15b370 msgr2=0x7f2a3c15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.551+0000 7f2a3b7fe640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a3c15b370 0x7f2a3c15e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.551+0000 7f2a3b7fe640 1 -- 172.21.15.97:0/1673200443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c158960 con 0x7f2a3c0730c0 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.551+0000 7f2a3b7fe640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a3c0730c0 0x7f2a3c14ccb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a30000c00 tx=0x7f2a300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.551+0000 7f2a38ff9640 1 -- 172.21.15.97:0/1673200443 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30035070 con 0x7f2a3c0730c0 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.551+0000 7f2a38ff9640 1 -- 172.21.15.97:0/1673200443 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a30004250 con 0x7f2a3c0730c0 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.551+0000 7f2a38ff9640 1 -- 172.21.15.97:0/1673200443 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3003a6f0 con 0x7f2a3c0730c0 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.551+0000 7f2a42560640 1 -- 172.21.15.97:0/1673200443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3c158c40 con 0x7f2a3c0730c0 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.551+0000 7f2a42560640 1 -- 172.21.15.97:0/1673200443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a3c159200 con 0x7f2a3c0730c0 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.552+0000 7f2a42560640 1 -- 172.21.15.97:0/1673200443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a3c068640 con 0x7f2a3c0730c0 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.552+0000 7f2a38ff9640 1 -- 172.21.15.97:0/1673200443 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2a30038020 con 0x7f2a3c0730c0 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.553+0000 7f2a38ff9640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2a2806c530 0x7f2a2806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:41.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.553+0000 7f2a3bfff640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2a2806c530 0x7f2a2806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:41.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.553+0000 7f2a38ff9640 1 -- 172.21.15.97:0/1673200443 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f2a300ad410 con 0x7f2a3c0730c0 2024-03-29T04:07:41.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.553+0000 7f2a3bfff640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2a2806c530 0x7f2a2806e9f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c000f60 tx=0x7f2a2c007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:41.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.557+0000 7f2a38ff9640 1 -- 172.21.15.97:0/1673200443 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2a30040070 con 0x7f2a3c0730c0 2024-03-29T04:07:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.734+0000 7f2a42560640 1 -- 172.21.15.97:0/1673200443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a3c0af760 con 0x7f2a3c0730c0 2024-03-29T04:07:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.735+0000 7f2a38ff9640 1 -- 172.21.15.97:0/1673200443 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16618 (secure 0 0 0) 0x7f2a3007ba30 con 0x7f2a3c0730c0 2024-03-29T04:07:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:41.715 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":19,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:40.761164+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.737+0000 7f2a42560640 1 -- 172.21.15.97:0/1673200443 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2a2806c530 msgr2=0x7f2a2806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.737+0000 7f2a42560640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2a2806c530 0x7f2a2806e9f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c000f60 tx=0x7f2a2c007c00 comp rx=0 tx=0).stop 2024-03-29T04:07:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.737+0000 7f2a42560640 1 -- 172.21.15.97:0/1673200443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a3c0730c0 msgr2=0x7f2a3c14ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.737+0000 7f2a42560640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a3c0730c0 0x7f2a3c14ccb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a30000c00 tx=0x7f2a300046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.737+0000 7f2a42560640 1 -- 172.21.15.97:0/1673200443 shutdown_connections 2024-03-29T04:07:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.737+0000 7f2a42560640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2a2806c530 0x7f2a2806e9f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.737+0000 7f2a42560640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a3c15b370 0x7f2a3c15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.737+0000 7f2a42560640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a3c0730c0 0x7f2a3c14ccb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.737+0000 7f2a42560640 1 --2- 172.21.15.97:0/1673200443 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a3c06f3f0 0x7f2a3c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.737+0000 7f2a42560640 1 -- 172.21.15.97:0/1673200443 >> 172.21.15.97:0/1673200443 conn(0x7f2a3c06b510 msgr2=0x7f2a3c0aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.737+0000 7f2a42560640 1 -- 172.21.15.97:0/1673200443 shutdown_connections 2024-03-29T04:07:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.737+0000 7f2a42560640 1 -- 172.21.15.97:0/1673200443 wait complete. 2024-03-29T04:07:41.728 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:41.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.848+0000 7f0971f32640 1 Processor -- start 2024-03-29T04:07:41.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.849+0000 7f0971f32640 1 -- start start 2024-03-29T04:07:41.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.849+0000 7f0971f32640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f096c15e930 0x7f096c0cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:41.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.849+0000 7f0971f32640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f096c0d3310 0x7f096c06cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:41.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.849+0000 7f0971f32640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f096c0cfb30 con 0x7f096c0cf670 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.849+0000 7f0971f32640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f096c06c780 con 0x7f096c0d3310 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.849+0000 7f0971f32640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f096c0cfe10 con 0x7f096c15e930 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.849+0000 7f096bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f096c0d3310 0x7f096c06cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.850+0000 7f0962ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f096c0cf670 0x7f096c0d2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50296/0 (socket says 172.21.15.97:50296) 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.850+0000 7f0962ffd640 1 -- 172.21.15.97:0/183489572 learned_addr learned my addr 172.21.15.97:0/183489572 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.850+0000 7f096b7fe640 1 --2- 172.21.15.97:0/183489572 >> v2:172.21.15.154:3300/0 conn(0x7f096c15e930 0x7f096c0cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.850+0000 7f096bfff640 1 -- 172.21.15.97:0/183489572 >> v2:172.21.15.154:3300/0 conn(0x7f096c15e930 msgr2=0x7f096c0cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.850+0000 7f096bfff640 1 --2- 172.21.15.97:0/183489572 >> v2:172.21.15.154:3300/0 conn(0x7f096c15e930 0x7f096c0cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.850+0000 7f096bfff640 1 -- 172.21.15.97:0/183489572 >> v1:172.21.15.97:6789/0 conn(0x7f096c0cf670 legacy=0x7f096c0d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.850+0000 7f096bfff640 1 -- 172.21.15.97:0/183489572 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096c0d00f0 con 0x7f096c0d3310 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.850+0000 7f096affd640 1 -- 172.21.15.97:0/183489572 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2337154887 0 0) 0x7f096c0cfb30 con 0x7f096c0cf670 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.851+0000 7f096bfff640 1 --2- 172.21.15.97:0/183489572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096c0d3310 0x7f096c06cc10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f096c06c400 tx=0x7f096402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5240501eaf981f48 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.851+0000 7f096affd640 1 -- 172.21.15.97:0/183489572 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964039070 con 0x7f096c0d3310 2024-03-29T04:07:41.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.851+0000 7f096affd640 1 -- 172.21.15.97:0/183489572 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096402cbf0 con 0x7f096c0d3310 2024-03-29T04:07:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.851+0000 7f096affd640 1 -- 172.21.15.97:0/183489572 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964034440 con 0x7f096c0d3310 2024-03-29T04:07:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.851+0000 7f0971f32640 1 -- 172.21.15.97:0/183489572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096c0d3310 msgr2=0x7f096c06cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.851+0000 7f0971f32640 1 --2- 172.21.15.97:0/183489572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096c0d3310 0x7f096c06cc10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f096c06c400 tx=0x7f096402be40 comp rx=0 tx=0).stop 2024-03-29T04:07:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.851+0000 7f0971f32640 1 -- 172.21.15.97:0/183489572 shutdown_connections 2024-03-29T04:07:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.851+0000 7f0971f32640 1 --2- 172.21.15.97:0/183489572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096c0d3310 0x7f096c06cc10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.851+0000 7f0971f32640 1 --2- 172.21.15.97:0/183489572 >> v2:172.21.15.154:3300/0 conn(0x7f096c15e930 0x7f096c0cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.851+0000 7f0971f32640 1 -- 172.21.15.97:0/183489572 >> 172.21.15.97:0/183489572 conn(0x7f096c08bfc0 msgr2=0x7f096c0d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.851+0000 7f0971f32640 1 -- 172.21.15.97:0/183489572 shutdown_connections 2024-03-29T04:07:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.851+0000 7f0971f32640 1 -- 172.21.15.97:0/183489572 wait complete. 2024-03-29T04:07:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.852+0000 7f0971f32640 1 Processor -- start 2024-03-29T04:07:41.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.853+0000 7f0971f32640 1 -- start start 2024-03-29T04:07:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.853+0000 7f0971f32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096c0cf670 0x7f096c15f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.853+0000 7f0971f32640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f096c0d3310 0x7f096c161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.853+0000 7f0971f32640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f096c15e930 0x7f096c1f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.853+0000 7f0971f32640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f096c15ee90 con 0x7f096c0cf670 2024-03-29T04:07:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.853+0000 7f0971f32640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f096c15ed10 con 0x7f096c15e930 2024-03-29T04:07:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.853+0000 7f0971f32640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f096c0b3c50 con 0x7f096c0d3310 2024-03-29T04:07:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.853+0000 7f096b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096c0cf670 0x7f096c15f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:41.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.853+0000 7f096b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096c0cf670 0x7f096c15f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38636/0 (socket says 172.21.15.97:38636) 2024-03-29T04:07:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.853+0000 7f096b7fe640 1 -- 172.21.15.97:0/794656675 learned_addr learned my addr 172.21.15.97:0/794656675 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.854+0000 7f096bfff640 1 --2- 172.21.15.97:0/794656675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f096c15e930 0x7f096c1f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.854+0000 7f096b7fe640 1 -- 172.21.15.97:0/794656675 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f096c0d3310 msgr2=0x7f096c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.854+0000 7f096b7fe640 1 --2- 172.21.15.97:0/794656675 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f096c0d3310 0x7f096c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.854+0000 7f096b7fe640 1 -- 172.21.15.97:0/794656675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f096c15e930 msgr2=0x7f096c1f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.854+0000 7f096b7fe640 1 --2- 172.21.15.97:0/794656675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f096c15e930 0x7f096c1f7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.854+0000 7f096b7fe640 1 -- 172.21.15.97:0/794656675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096c15f6c0 con 0x7f096c0cf670 2024-03-29T04:07:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.854+0000 7f096b7fe640 1 --2- 172.21.15.97:0/794656675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096c0cf670 0x7f096c15f070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f095c00d5f0 tx=0x7f095c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.854+0000 7f0968ff9640 1 -- 172.21.15.97:0/794656675 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c0125f0 con 0x7f096c0cf670 2024-03-29T04:07:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.854+0000 7f0968ff9640 1 -- 172.21.15.97:0/794656675 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c019070 con 0x7f096c0cf670 2024-03-29T04:07:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.854+0000 7f0971f32640 1 -- 172.21.15.97:0/794656675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096c15f9a0 con 0x7f096c0cf670 2024-03-29T04:07:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.854+0000 7f0968ff9640 1 -- 172.21.15.97:0/794656675 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c0129d0 con 0x7f096c0cf670 2024-03-29T04:07:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.854+0000 7f0971f32640 1 -- 172.21.15.97:0/794656675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096c15ff60 con 0x7f096c0cf670 2024-03-29T04:07:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.855+0000 7f0971f32640 1 -- 172.21.15.97:0/794656675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096c068640 con 0x7f096c0cf670 2024-03-29T04:07:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.855+0000 7f0968ff9640 1 -- 172.21.15.97:0/794656675 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f095c012b70 con 0x7f096c0cf670 2024-03-29T04:07:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.856+0000 7f0968ff9640 1 --2- 172.21.15.97:0/794656675 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f094c06c530 0x7f094c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.856+0000 7f0968ff9640 1 -- 172.21.15.97:0/794656675 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f095c08bac0 con 0x7f096c0cf670 2024-03-29T04:07:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.856+0000 7f0962ffd640 1 --2- 172.21.15.97:0/794656675 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f094c06c530 0x7f094c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.856+0000 7f0962ffd640 1 --2- 172.21.15.97:0/794656675 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f094c06c530 0x7f094c06e9f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0954005880 tx=0x7f0954007950 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:41.860+0000 7f0968ff9640 1 -- 172.21.15.97:0/794656675 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f095c05a0e0 con 0x7f096c0cf670 2024-03-29T04:07:42.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.035+0000 7f0971f32640 1 -- 172.21.15.97:0/794656675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f096c0cea50 con 0x7f096c0cf670 2024-03-29T04:07:42.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.036+0000 7f0968ff9640 1 -- 172.21.15.97:0/794656675 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16618 (secure 0 0 0) 0x7f095c0599e0 con 0x7f096c0cf670 2024-03-29T04:07:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:42.016 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":19,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:40.761164+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.038+0000 7f0971f32640 1 -- 172.21.15.97:0/794656675 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f094c06c530 msgr2=0x7f094c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.038+0000 7f0971f32640 1 --2- 172.21.15.97:0/794656675 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f094c06c530 0x7f094c06e9f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0954005880 tx=0x7f0954007950 comp rx=0 tx=0).stop 2024-03-29T04:07:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.038+0000 7f0971f32640 1 -- 172.21.15.97:0/794656675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096c0cf670 msgr2=0x7f096c15f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.038+0000 7f0971f32640 1 --2- 172.21.15.97:0/794656675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096c0cf670 0x7f096c15f070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f095c00d5f0 tx=0x7f095c00dab0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.038+0000 7f0971f32640 1 -- 172.21.15.97:0/794656675 shutdown_connections 2024-03-29T04:07:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.038+0000 7f0971f32640 1 --2- 172.21.15.97:0/794656675 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f094c06c530 0x7f094c06e9f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.038+0000 7f0971f32640 1 --2- 172.21.15.97:0/794656675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f096c15e930 0x7f096c1f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.038+0000 7f0971f32640 1 --2- 172.21.15.97:0/794656675 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f096c0d3310 0x7f096c161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.038+0000 7f0971f32640 1 --2- 172.21.15.97:0/794656675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f096c0cf670 0x7f096c15f070 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.038+0000 7f0971f32640 1 -- 172.21.15.97:0/794656675 >> 172.21.15.97:0/794656675 conn(0x7f096c08bfc0 msgr2=0x7f096c0b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.038+0000 7f0971f32640 1 -- 172.21.15.97:0/794656675 shutdown_connections 2024-03-29T04:07:42.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.038+0000 7f0971f32640 1 -- 172.21.15.97:0/794656675 wait complete. 2024-03-29T04:07:42.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T04:07:42.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.152+0000 7f4916ed5640 1 Processor -- start 2024-03-29T04:07:42.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.152+0000 7f4916ed5640 1 -- start start 2024-03-29T04:07:42.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4916ed5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49100ce680 0x7f491009b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:42.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4916ed5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f491009b140 0x7f49100cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:42.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4916ed5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49100d0c80 con 0x7f49100cea60 2024-03-29T04:07:42.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4916ed5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f49100d0a30 con 0x7f491009b140 2024-03-29T04:07:42.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4916ed5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4910070650 con 0x7f49100ce680 2024-03-29T04:07:42.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4915472640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f491009b140 0x7f49100cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:42.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4915472640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f491009b140 0x7f49100cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38642/0 (socket says 172.21.15.97:38642) 2024-03-29T04:07:42.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4915472640 1 -- 172.21.15.97:0/3335433716 learned_addr learned my addr 172.21.15.97:0/3335433716 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:42.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4914c71640 1 --2- 172.21.15.97:0/3335433716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49100ce680 0x7f491009b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:42.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4915472640 1 -- 172.21.15.97:0/3335433716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49100ce680 msgr2=0x7f491009b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:42.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4915472640 1 --2- 172.21.15.97:0/3335433716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49100ce680 0x7f491009b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4915472640 1 -- 172.21.15.97:0/3335433716 >> v1:172.21.15.154:6789/0 conn(0x7f49100cea60 legacy=0x7f491009aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:42.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4915472640 1 -- 172.21.15.97:0/3335433716 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49100d0f60 con 0x7f491009b140 2024-03-29T04:07:42.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.153+0000 7f4914c71640 1 --2- 172.21.15.97:0/3335433716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49100ce680 0x7f491009b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:42.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.154+0000 7f4915472640 1 --2- 172.21.15.97:0/3335433716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f491009b140 0x7f49100cce30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4908009e50 tx=0x7f490802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205b54059665be94 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:42.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.154+0000 7f490f7fe640 1 -- 172.21.15.97:0/3335433716 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490802ca70 con 0x7f491009b140 2024-03-29T04:07:42.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.154+0000 7f490f7fe640 1 -- 172.21.15.97:0/3335433716 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490802cc10 con 0x7f491009b140 2024-03-29T04:07:42.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.154+0000 7f490f7fe640 1 -- 172.21.15.97:0/3335433716 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49080343f0 con 0x7f491009b140 2024-03-29T04:07:42.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.154+0000 7f4916ed5640 1 -- 172.21.15.97:0/3335433716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f491009b140 msgr2=0x7f49100cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:42.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.154+0000 7f4916ed5640 1 --2- 172.21.15.97:0/3335433716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f491009b140 0x7f49100cce30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4908009e50 tx=0x7f490802bed0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.154+0000 7f4916ed5640 1 -- 172.21.15.97:0/3335433716 shutdown_connections 2024-03-29T04:07:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.154+0000 7f4916ed5640 1 --2- 172.21.15.97:0/3335433716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f491009b140 0x7f49100cce30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.154+0000 7f4916ed5640 1 --2- 172.21.15.97:0/3335433716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49100ce680 0x7f491009b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.154+0000 7f4916ed5640 1 -- 172.21.15.97:0/3335433716 >> 172.21.15.97:0/3335433716 conn(0x7f49100b3f40 msgr2=0x7f49100b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.155+0000 7f4916ed5640 1 -- 172.21.15.97:0/3335433716 shutdown_connections 2024-03-29T04:07:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.155+0000 7f4916ed5640 1 -- 172.21.15.97:0/3335433716 wait complete. 2024-03-29T04:07:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.156+0000 7f4916ed5640 1 Processor -- start 2024-03-29T04:07:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.156+0000 7f4916ed5640 1 -- start start 2024-03-29T04:07:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4916ed5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f491009b140 0x7f49100c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4916ed5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49100ce680 0x7f49100c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4916ed5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49100cea60 0x7f49100c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4916ed5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4910070ae0 con 0x7f491009b140 2024-03-29T04:07:42.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4916ed5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f491009bcb0 con 0x7f49100cea60 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4916ed5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4910070c60 con 0x7f49100ce680 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4914c71640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f491009b140 0x7f49100c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4914c71640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f491009b140 0x7f49100c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38646/0 (socket says 172.21.15.97:38646) 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4914c71640 1 -- 172.21.15.97:0/2945361843 learned_addr learned my addr 172.21.15.97:0/2945361843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4914c71640 1 -- 172.21.15.97:0/2945361843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49100ce680 msgr2=0x7f49100c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4914c71640 1 --2- 172.21.15.97:0/2945361843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49100ce680 0x7f49100c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4914c71640 1 -- 172.21.15.97:0/2945361843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49100cea60 msgr2=0x7f49100c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4914c71640 1 --2- 172.21.15.97:0/2945361843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49100cea60 0x7f49100c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4914c71640 1 -- 172.21.15.97:0/2945361843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49100c7910 con 0x7f491009b140 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4914c71640 1 --2- 172.21.15.97:0/2945361843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f491009b140 0x7f49100c9520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f490000d800 tx=0x7f490000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f490d7fa640 1 -- 172.21.15.97:0/2945361843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900004030 con 0x7f491009b140 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f490d7fa640 1 -- 172.21.15.97:0/2945361843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49000041d0 con 0x7f491009b140 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4916ed5640 1 -- 172.21.15.97:0/2945361843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49100c7bf0 con 0x7f491009b140 2024-03-29T04:07:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f490d7fa640 1 -- 172.21.15.97:0/2945361843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49000096a0 con 0x7f491009b140 2024-03-29T04:07:42.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.157+0000 7f4916ed5640 1 -- 172.21.15.97:0/2945361843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49100c81b0 con 0x7f491009b140 2024-03-29T04:07:42.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.158+0000 7f4916ed5640 1 -- 172.21.15.97:0/2945361843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49100a0520 con 0x7f491009b140 2024-03-29T04:07:42.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.158+0000 7f490d7fa640 1 -- 172.21.15.97:0/2945361843 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4900009840 con 0x7f491009b140 2024-03-29T04:07:42.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.159+0000 7f490d7fa640 1 --2- 172.21.15.97:0/2945361843 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f48fc06c530 0x7f48fc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:42.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.159+0000 7f490d7fa640 1 -- 172.21.15.97:0/2945361843 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f490008b940 con 0x7f491009b140 2024-03-29T04:07:42.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.159+0000 7f490ffff640 1 --2- 172.21.15.97:0/2945361843 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f48fc06c530 0x7f48fc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:42.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.159+0000 7f490ffff640 1 --2- 172.21.15.97:0/2945361843 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f48fc06c530 0x7f48fc06e9f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4904000f60 tx=0x7f49040079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:42.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.163+0000 7f490d7fa640 1 -- 172.21.15.97:0/2945361843 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4900059ff0 con 0x7f491009b140 2024-03-29T04:07:42.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.376+0000 7f4916ed5640 1 -- 172.21.15.97:0/2945361843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f491004f9d0 con 0x7f491009b140 2024-03-29T04:07:42.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.427+0000 7f490d7fa640 1 -- 172.21.15.97:0/2945361843 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7f49000598f0 con 0x7f491009b140 2024-03-29T04:07:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.429+0000 7f4916ed5640 1 -- 172.21.15.97:0/2945361843 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f48fc06c530 msgr2=0x7f48fc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.429+0000 7f4916ed5640 1 --2- 172.21.15.97:0/2945361843 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f48fc06c530 0x7f48fc06e9f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4904000f60 tx=0x7f49040079d0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.429+0000 7f4916ed5640 1 -- 172.21.15.97:0/2945361843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f491009b140 msgr2=0x7f49100c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.429+0000 7f4916ed5640 1 --2- 172.21.15.97:0/2945361843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f491009b140 0x7f49100c9520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f490000d800 tx=0x7f490000dcc0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.429+0000 7f4916ed5640 1 -- 172.21.15.97:0/2945361843 shutdown_connections 2024-03-29T04:07:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.429+0000 7f4916ed5640 1 --2- 172.21.15.97:0/2945361843 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f48fc06c530 0x7f48fc06e9f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.429+0000 7f4916ed5640 1 --2- 172.21.15.97:0/2945361843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49100cea60 0x7f49100c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.429+0000 7f4916ed5640 1 --2- 172.21.15.97:0/2945361843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49100ce680 0x7f49100c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.429+0000 7f4916ed5640 1 --2- 172.21.15.97:0/2945361843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f491009b140 0x7f49100c9520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.429+0000 7f4916ed5640 1 -- 172.21.15.97:0/2945361843 >> 172.21.15.97:0/2945361843 conn(0x7f49100b3f40 msgr2=0x7f49100b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.429+0000 7f4916ed5640 1 -- 172.21.15.97:0/2945361843 shutdown_connections 2024-03-29T04:07:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.429+0000 7f4916ed5640 1 -- 172.21.15.97:0/2945361843 wait complete. 2024-03-29T04:07:42.419 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T04:07:42.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T04:07:42.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.543+0000 7fa4a9996640 1 Processor -- start 2024-03-29T04:07:42.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.543+0000 7fa4a9996640 1 -- start start 2024-03-29T04:07:42.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.543+0000 7fa4a9996640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa4a415e920 0x7fa4a40d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:42.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.543+0000 7fa4a9996640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4a40cfde0 0x7fa4a40d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:42.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.543+0000 7fa4a9996640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa4a40d0f90 0x7fa4a40d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:42.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.543+0000 7fa4a9996640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa4a40d3780 con 0x7fa4a40d0f90 2024-03-29T04:07:42.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.543+0000 7fa4a9996640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fa4a415ed00 con 0x7fa4a415e920 2024-03-29T04:07:42.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.543+0000 7fa4a9996640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a40d21e0 con 0x7fa4a40cfde0 2024-03-29T04:07:42.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.544+0000 7fa4a37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa4a40d0f90 0x7fa4a40d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:42.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.544+0000 7fa4a37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa4a40d0f90 0x7fa4a40d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38650/0 (socket says 172.21.15.97:38650) 2024-03-29T04:07:42.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.544+0000 7fa4a37fe640 1 -- 172.21.15.97:0/926421159 learned_addr learned my addr 172.21.15.97:0/926421159 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.544+0000 7fa4a27fc640 1 --2- 172.21.15.97:0/926421159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4a40cfde0 0x7fa4a40d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.544+0000 7fa4a2ffd640 1 --2- 172.21.15.97:0/926421159 >> v2:172.21.15.154:3300/0 conn(0x7fa4a415e920 0x7fa4a40d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.544+0000 7fa4a37fe640 1 -- 172.21.15.97:0/926421159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4a40cfde0 msgr2=0x7fa4a40d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.544+0000 7fa4a37fe640 1 --2- 172.21.15.97:0/926421159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4a40cfde0 0x7fa4a40d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.544+0000 7fa4a37fe640 1 -- 172.21.15.97:0/926421159 >> v2:172.21.15.154:3300/0 conn(0x7fa4a415e920 msgr2=0x7fa4a40d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.544+0000 7fa4a37fe640 1 --2- 172.21.15.97:0/926421159 >> v2:172.21.15.154:3300/0 conn(0x7fa4a415e920 0x7fa4a40d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.544+0000 7fa4a37fe640 1 -- 172.21.15.97:0/926421159 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a40d2480 con 0x7fa4a40d0f90 2024-03-29T04:07:42.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.545+0000 7fa4a37fe640 1 --2- 172.21.15.97:0/926421159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a40d0f90 0x7fa4a40d1420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009e50 tx=0x7fa49c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c2a80dbc473930 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:42.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.545+0000 7fa4a1ffb640 1 -- 172.21.15.97:0/926421159 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c02ca70 con 0x7fa4a40d0f90 2024-03-29T04:07:42.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.545+0000 7fa4a1ffb640 1 -- 172.21.15.97:0/926421159 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa49c02cc10 con 0x7fa4a40d0f90 2024-03-29T04:07:42.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.545+0000 7fa4a1ffb640 1 -- 172.21.15.97:0/926421159 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c0343f0 con 0x7fa4a40d0f90 2024-03-29T04:07:42.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.545+0000 7fa4a9996640 1 -- 172.21.15.97:0/926421159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a40d0f90 msgr2=0x7fa4a40d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:42.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.545+0000 7fa4a9996640 1 --2- 172.21.15.97:0/926421159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a40d0f90 0x7fa4a40d1420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009e50 tx=0x7fa49c02bed0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.545+0000 7fa4a9996640 1 -- 172.21.15.97:0/926421159 shutdown_connections 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.545+0000 7fa4a9996640 1 --2- 172.21.15.97:0/926421159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a40d0f90 0x7fa4a40d1420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.545+0000 7fa4a9996640 1 --2- 172.21.15.97:0/926421159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4a40cfde0 0x7fa4a40d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.545+0000 7fa4a9996640 1 --2- 172.21.15.97:0/926421159 >> v2:172.21.15.154:3300/0 conn(0x7fa4a415e920 0x7fa4a40d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.545+0000 7fa4a9996640 1 -- 172.21.15.97:0/926421159 >> 172.21.15.97:0/926421159 conn(0x7fa4a408bfc0 msgr2=0x7fa4a40d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.545+0000 7fa4a9996640 1 -- 172.21.15.97:0/926421159 shutdown_connections 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.545+0000 7fa4a9996640 1 -- 172.21.15.97:0/926421159 wait complete. 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.546+0000 7fa4a9996640 1 Processor -- start 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.547+0000 7fa4a9996640 1 -- start start 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.547+0000 7fa4a9996640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a40cfde0 0x7fa4a40cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.547+0000 7fa4a9996640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4a40d0f90 0x7fa4a40cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.547+0000 7fa4a9996640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a415e920 0x7fa4a414bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.547+0000 7fa4a9996640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a40d2d20 con 0x7fa4a40cfde0 2024-03-29T04:07:42.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.547+0000 7fa4a9996640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a40d2a40 con 0x7fa4a415e920 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.547+0000 7fa4a9996640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a40d4ef0 con 0x7fa4a40d0f90 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.547+0000 7fa4a2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a40cfde0 0x7fa4a40cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.547+0000 7fa4a2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a40cfde0 0x7fa4a40cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38660/0 (socket says 172.21.15.97:38660) 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.547+0000 7fa4a2ffd640 1 -- 172.21.15.97:0/4073290685 learned_addr learned my addr 172.21.15.97:0/4073290685 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.547+0000 7fa4a37fe640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a415e920 0x7fa4a414bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.547+0000 7fa4a27fc640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4a40d0f90 0x7fa4a40cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.548+0000 7fa4a2ffd640 1 -- 172.21.15.97:0/4073290685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4a40d0f90 msgr2=0x7fa4a40cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.548+0000 7fa4a2ffd640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4a40d0f90 0x7fa4a40cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.548+0000 7fa4a2ffd640 1 -- 172.21.15.97:0/4073290685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a415e920 msgr2=0x7fa4a414bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.548+0000 7fa4a2ffd640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a415e920 0x7fa4a414bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.548+0000 7fa4a2ffd640 1 -- 172.21.15.97:0/4073290685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a414c1f0 con 0x7fa4a40cfde0 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.548+0000 7fa4a37fe640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a415e920 0x7fa4a414bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.548+0000 7fa4a27fc640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4a40d0f90 0x7fa4a40cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.548+0000 7fa4a2ffd640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a40cfde0 0x7fa4a40cae00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa49400e9e0 tx=0x7fa49400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.548+0000 7fa48bfff640 1 -- 172.21.15.97:0/4073290685 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494017070 con 0x7fa4a40cfde0 2024-03-29T04:07:42.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.548+0000 7fa48bfff640 1 -- 172.21.15.97:0/4073290685 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa49400c970 con 0x7fa4a40cfde0 2024-03-29T04:07:42.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.548+0000 7fa48bfff640 1 -- 172.21.15.97:0/4073290685 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49400cd60 con 0x7fa4a40cfde0 2024-03-29T04:07:42.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.548+0000 7fa4a9996640 1 -- 172.21.15.97:0/4073290685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a40cbe50 con 0x7fa4a40cfde0 2024-03-29T04:07:42.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.548+0000 7fa4a9996640 1 -- 172.21.15.97:0/4073290685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a40cc410 con 0x7fa4a40cfde0 2024-03-29T04:07:42.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.549+0000 7fa4a9996640 1 -- 172.21.15.97:0/4073290685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a40a2560 con 0x7fa4a40cfde0 2024-03-29T04:07:42.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.549+0000 7fa48bfff640 1 -- 172.21.15.97:0/4073290685 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa49401a070 con 0x7fa4a40cfde0 2024-03-29T04:07:42.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.550+0000 7fa48bfff640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa49006c530 0x7fa49006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:42.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.550+0000 7fa48bfff640 1 -- 172.21.15.97:0/4073290685 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa49408b670 con 0x7fa4a40cfde0 2024-03-29T04:07:42.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.550+0000 7fa4a27fc640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa49006c530 0x7fa49006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:42.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.550+0000 7fa4a27fc640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa49006c530 0x7fa49006e9f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40d04a0 tx=0x7fa498008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:42.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.554+0000 7fa48bfff640 1 -- 172.21.15.97:0/4073290685 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa494059c90 con 0x7fa4a40cfde0 2024-03-29T04:07:42.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:42.770+0000 7fa4a9996640 1 -- 172.21.15.97:0/4073290685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa4a415e0d0 con 0x7fa4a40cfde0 2024-03-29T04:07:43.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.430+0000 7fa48bfff640 1 -- 172.21.15.97:0/4073290685 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7fa494059590 con 0x7fa4a40cfde0 2024-03-29T04:07:43.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.433+0000 7fa4a9996640 1 -- 172.21.15.97:0/4073290685 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa49006c530 msgr2=0x7fa49006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:43.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.433+0000 7fa4a9996640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa49006c530 0x7fa49006e9f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40d04a0 tx=0x7fa498008040 comp rx=0 tx=0).stop 2024-03-29T04:07:43.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.433+0000 7fa4a9996640 1 -- 172.21.15.97:0/4073290685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a40cfde0 msgr2=0x7fa4a40cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:43.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.433+0000 7fa4a9996640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a40cfde0 0x7fa4a40cae00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa49400e9e0 tx=0x7fa49400eea0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.433+0000 7fa4a9996640 1 -- 172.21.15.97:0/4073290685 shutdown_connections 2024-03-29T04:07:43.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.433+0000 7fa4a9996640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa49006c530 0x7fa49006e9f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.433+0000 7fa4a9996640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4a415e920 0x7fa4a414bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.433+0000 7fa4a9996640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa4a40d0f90 0x7fa4a40cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.433+0000 7fa4a9996640 1 --2- 172.21.15.97:0/4073290685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a40cfde0 0x7fa4a40cae00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.433+0000 7fa4a9996640 1 -- 172.21.15.97:0/4073290685 >> 172.21.15.97:0/4073290685 conn(0x7fa4a408bfc0 msgr2=0x7fa4a40b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:43.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.434+0000 7fa4a9996640 1 -- 172.21.15.97:0/4073290685 shutdown_connections 2024-03-29T04:07:43.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.434+0000 7fa4a9996640 1 -- 172.21.15.97:0/4073290685 wait complete. 2024-03-29T04:07:43.425 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T04:07:43.425 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:07:43.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.546+0000 7f3b15e4a640 1 Processor -- start 2024-03-29T04:07:43.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.546+0000 7f3b15e4a640 1 -- start start 2024-03-29T04:07:43.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b15e4a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3b1015e6e0 0x7f3b1009b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:43.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b15e4a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b100d0660 0x7f3b101623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b15e4a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b100d1a10 con 0x7f3b101628e0 2024-03-29T04:07:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b15e4a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f3b100d0e90 con 0x7f3b100d0660 2024-03-29T04:07:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b15e4a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f3b100d1170 con 0x7f3b1015e6e0 2024-03-29T04:07:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b0effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b100d0660 0x7f3b101623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b0effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b100d0660 0x7f3b101623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38662/0 (socket says 172.21.15.97:38662) 2024-03-29T04:07:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b0ffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3b101628e0 0x7f3b100b30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50310/0 (socket says 172.21.15.97:50310) 2024-03-29T04:07:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b0effd640 1 -- 172.21.15.97:0/1517615532 learned_addr learned my addr 172.21.15.97:0/1517615532 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b0f7fe640 1 --2- 172.21.15.97:0/1517615532 >> v2:172.21.15.154:3300/0 conn(0x7f3b1015e6e0 0x7f3b1009b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b0effd640 1 -- 172.21.15.97:0/1517615532 >> v2:172.21.15.154:3300/0 conn(0x7f3b1015e6e0 msgr2=0x7f3b1009b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b0effd640 1 --2- 172.21.15.97:0/1517615532 >> v2:172.21.15.154:3300/0 conn(0x7f3b1015e6e0 0x7f3b1009b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b0effd640 1 -- 172.21.15.97:0/1517615532 >> v1:172.21.15.97:6789/0 conn(0x7f3b101628e0 legacy=0x7f3b100b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b0effd640 1 -- 172.21.15.97:0/1517615532 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b100d1450 con 0x7f3b100d0660 2024-03-29T04:07:43.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.547+0000 7f3b0e7fc640 1 -- 172.21.15.97:0/1517615532 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 50511267 0 0) 0x7f3b100d1a10 con 0x7f3b101628e0 2024-03-29T04:07:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.548+0000 7f3b0effd640 1 --2- 172.21.15.97:0/1517615532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b100d0660 0x7f3b101623a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3b10072bc0 tx=0x7f3b0402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c231b38f9143adaa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.548+0000 7f3b0e7fc640 1 -- 172.21.15.97:0/1517615532 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04034070 con 0x7f3b100d0660 2024-03-29T04:07:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.548+0000 7f3b0e7fc640 1 -- 172.21.15.97:0/1517615532 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b04030860 con 0x7f3b100d0660 2024-03-29T04:07:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.548+0000 7f3b0e7fc640 1 -- 172.21.15.97:0/1517615532 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04030c30 con 0x7f3b100d0660 2024-03-29T04:07:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.549+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1517615532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b100d0660 msgr2=0x7f3b101623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.549+0000 7f3b15e4a640 1 --2- 172.21.15.97:0/1517615532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b100d0660 0x7f3b101623a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3b10072bc0 tx=0x7f3b0402b9a0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.549+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1517615532 shutdown_connections 2024-03-29T04:07:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.549+0000 7f3b15e4a640 1 --2- 172.21.15.97:0/1517615532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b100d0660 0x7f3b101623a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.549+0000 7f3b15e4a640 1 --2- 172.21.15.97:0/1517615532 >> v2:172.21.15.154:3300/0 conn(0x7f3b1015e6e0 0x7f3b1009b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.549+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1517615532 >> 172.21.15.97:0/1517615532 conn(0x7f3b100b3f10 msgr2=0x7f3b100b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:43.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.549+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1517615532 shutdown_connections 2024-03-29T04:07:43.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.549+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1517615532 wait complete. 2024-03-29T04:07:43.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.550+0000 7f3b15e4a640 1 Processor -- start 2024-03-29T04:07:43.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.550+0000 7f3b15e4a640 1 -- start start 2024-03-29T04:07:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b15e4a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b100d0660 0x7f3b10159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b15e4a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b1015e6e0 0x7f3b100b6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b15e4a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b101628e0 0x7f3b100707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b15e4a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1006d910 con 0x7f3b101628e0 2024-03-29T04:07:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b15e4a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3b100b1d80 con 0x7f3b1015e6e0 2024-03-29T04:07:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b15e4a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3b100b6e50 con 0x7f3b100d0660 2024-03-29T04:07:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b0ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b101628e0 0x7f3b100707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b0ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b101628e0 0x7f3b100707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38666/0 (socket says 172.21.15.97:38666) 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b0ffff640 1 -- 172.21.15.97:0/1103454659 learned_addr learned my addr 172.21.15.97:0/1103454659 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b0f7fe640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b100d0660 0x7f3b10159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b0effd640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b1015e6e0 0x7f3b100b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b0ffff640 1 -- 172.21.15.97:0/1103454659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b100d0660 msgr2=0x7f3b10159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b0ffff640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b100d0660 0x7f3b10159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b0ffff640 1 -- 172.21.15.97:0/1103454659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b1015e6e0 msgr2=0x7f3b100b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b0ffff640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b1015e6e0 0x7f3b100b6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b0ffff640 1 -- 172.21.15.97:0/1103454659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b100b7130 con 0x7f3b101628e0 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b0f7fe640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b100d0660 0x7f3b10159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b0effd640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b1015e6e0 0x7f3b100b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b0ffff640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b101628e0 0x7f3b100707c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800c730 tx=0x7f3b0800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3af7fff640 1 -- 172.21.15.97:0/1103454659 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08018070 con 0x7f3b101628e0 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3af7fff640 1 -- 172.21.15.97:0/1103454659 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b0800ce40 con 0x7f3b101628e0 2024-03-29T04:07:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.551+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1103454659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b100b7410 con 0x7f3b101628e0 2024-03-29T04:07:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.552+0000 7f3af7fff640 1 -- 172.21.15.97:0/1103454659 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0800a860 con 0x7f3b101628e0 2024-03-29T04:07:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.552+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1103454659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b100b79d0 con 0x7f3b101628e0 2024-03-29T04:07:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.553+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1103454659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b10068640 con 0x7f3b101628e0 2024-03-29T04:07:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.553+0000 7f3af7fff640 1 -- 172.21.15.97:0/1103454659 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3b0801b070 con 0x7f3b101628e0 2024-03-29T04:07:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.553+0000 7f3af7fff640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3afc06c530 0x7f3afc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.553+0000 7f3af7fff640 1 -- 172.21.15.97:0/1103454659 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3b0808acd0 con 0x7f3b101628e0 2024-03-29T04:07:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.553+0000 7f3b0f7fe640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3afc06c530 0x7f3afc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.554+0000 7f3b0f7fe640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3afc06c530 0x7f3afc06e9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3b100b3290 tx=0x7f3b00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:43.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.558+0000 7f3af7fff640 1 -- 172.21.15.97:0/1103454659 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3b080592f0 con 0x7f3b101628e0 2024-03-29T04:07:43.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.769+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1103454659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b100d2180 con 0x7f3b101628e0 2024-03-29T04:07:43.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.770+0000 7f3af7fff640 1 -- 172.21.15.97:0/1103454659 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7f3b08058bf0 con 0x7f3b101628e0 2024-03-29T04:07:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:43.750 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1127043830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127043830},{"type":"v1","addr":"172.21.15.154:6834","nonce":1127043830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3767188750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3767188750},{"type":"v1","addr":"172.21.15.97:6835","nonce":3767188750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3651340478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3651340478},{"type":"v1","addr":"172.21.15.97:6837","nonce":3651340478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:07:40.761466+0000","modified":"2024-03-29T04:07:42.771721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3669418265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3669418265},{"type":"v1","addr":"172.21.15.154:6835","nonce":3669418265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-29T04:07:43.750 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-03-29T04:07:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.771+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1103454659 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3afc06c530 msgr2=0x7f3afc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.771+0000 7f3b15e4a640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3afc06c530 0x7f3afc06e9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3b100b3290 tx=0x7f3b00008040 comp rx=0 tx=0).stop 2024-03-29T04:07:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.771+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1103454659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b101628e0 msgr2=0x7f3b100707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.771+0000 7f3b15e4a640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b101628e0 0x7f3b100707c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800c730 tx=0x7f3b0800cbf0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.772+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1103454659 shutdown_connections 2024-03-29T04:07:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.772+0000 7f3b15e4a640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3afc06c530 0x7f3afc06e9f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.772+0000 7f3b15e4a640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b101628e0 0x7f3b100707c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.772+0000 7f3b15e4a640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b1015e6e0 0x7f3b100b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.772+0000 7f3b15e4a640 1 --2- 172.21.15.97:0/1103454659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b100d0660 0x7f3b10159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.772+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1103454659 >> 172.21.15.97:0/1103454659 conn(0x7f3b100b3f10 msgr2=0x7f3b1015cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.772+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1103454659 shutdown_connections 2024-03-29T04:07:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.772+0000 7f3b15e4a640 1 -- 172.21.15.97:0/1103454659 wait complete. 2024-03-29T04:07:43.762 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:43.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.884+0000 7f89c531c640 1 Processor -- start 2024-03-29T04:07:43.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.884+0000 7f89c531c640 1 -- start start 2024-03-29T04:07:43.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.884+0000 7f89c531c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89c00b3af0 0x7f89c006c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:43.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.884+0000 7f89c531c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89c006f1c0 con 0x7f89c006ca60 2024-03-29T04:07:43.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.884+0000 7f89c531c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89c006f4a0 con 0x7f89c015b530 2024-03-29T04:07:43.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.884+0000 7f89c531c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f89c006ceb0 con 0x7f89c00b3af0 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.885+0000 7f89bf7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f89c006ca60 0x7f89c00ced00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50324/0 (socket says 172.21.15.97:50324) 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.885+0000 7f89bf7fe640 1 -- 172.21.15.97:0/745823248 learned_addr learned my addr 172.21.15.97:0/745823248 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.885+0000 7f89b7fff640 1 --2- 172.21.15.97:0/745823248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89c00b3af0 0x7f89c006c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.885+0000 7f89be7fc640 1 -- 172.21.15.97:0/745823248 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2443372605 0 0) 0x7f89c006f1c0 con 0x7f89c006ca60 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.885+0000 7f89be7fc640 1 -- 172.21.15.97:0/745823248 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89ac003520 con 0x7f89c006ca60 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.885+0000 7f89be7fc640 1 -- 172.21.15.97:0/745823248 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3172713895 0 0) 0x7f89c006f4a0 con 0x7f89c015b530 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.885+0000 7f89be7fc640 1 -- 172.21.15.97:0/745823248 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89c006f1c0 con 0x7f89c015b530 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.885+0000 7f89b7fff640 1 -- 172.21.15.97:0/745823248 >> v1:172.21.15.154:6789/0 conn(0x7f89c015b530 legacy=0x7f89c0072dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.885+0000 7f89b7fff640 1 -- 172.21.15.97:0/745823248 >> v1:172.21.15.97:6789/0 conn(0x7f89c006ca60 legacy=0x7f89c00ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.885+0000 7f89b7fff640 1 -- 172.21.15.97:0/745823248 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c006f780 con 0x7f89c00b3af0 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.886+0000 7f89b7fff640 1 --2- 172.21.15.97:0/745823248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89c00b3af0 0x7f89c006c520 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f89a80097d0 tx=0x7f89a802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9a2a21ba1fa9fd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.886+0000 7f89be7fc640 1 -- 172.21.15.97:0/745823248 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8035020 con 0x7f89c00b3af0 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.886+0000 7f89be7fc640 1 -- 172.21.15.97:0/745823248 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a8031800 con 0x7f89c00b3af0 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.886+0000 7f89be7fc640 1 -- 172.21.15.97:0/745823248 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8031bb0 con 0x7f89c00b3af0 2024-03-29T04:07:43.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.886+0000 7f89c531c640 1 -- 172.21.15.97:0/745823248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89c00b3af0 msgr2=0x7f89c006c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:43.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.886+0000 7f89c531c640 1 --2- 172.21.15.97:0/745823248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89c00b3af0 0x7f89c006c520 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f89a80097d0 tx=0x7f89a802b8a0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.886+0000 7f89c531c640 1 -- 172.21.15.97:0/745823248 shutdown_connections 2024-03-29T04:07:43.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.886+0000 7f89c531c640 1 --2- 172.21.15.97:0/745823248 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89c00b3af0 0x7f89c006c520 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.886+0000 7f89c531c640 1 -- 172.21.15.97:0/745823248 >> 172.21.15.97:0/745823248 conn(0x7f89c0072050 msgr2=0x7f89c0072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:43.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.886+0000 7f89c531c640 1 -- 172.21.15.97:0/745823248 shutdown_connections 2024-03-29T04:07:43.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.887+0000 7f89c531c640 1 -- 172.21.15.97:0/745823248 wait complete. 2024-03-29T04:07:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.888+0000 7f89c531c640 1 Processor -- start 2024-03-29T04:07:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.888+0000 7f89c531c640 1 -- start start 2024-03-29T04:07:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.888+0000 7f89c531c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c006ca60 0x7f89c0149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.888+0000 7f89c531c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89c00b3af0 0x7f89c01599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.888+0000 7f89c531c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89c015b530 0x7f89c00b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.888+0000 7f89c531c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f89c00ce7b0 con 0x7f89c006ca60 2024-03-29T04:07:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.888+0000 7f89c531c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f89c006fd40 con 0x7f89c015b530 2024-03-29T04:07:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.888+0000 7f89c531c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f89c015b910 con 0x7f89c00b3af0 2024-03-29T04:07:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.888+0000 7f89beffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c006ca60 0x7f89c0149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.888+0000 7f89beffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c006ca60 0x7f89c0149bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38680/0 (socket says 172.21.15.97:38680) 2024-03-29T04:07:43.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.888+0000 7f89beffd640 1 -- 172.21.15.97:0/4084436431 learned_addr learned my addr 172.21.15.97:0/4084436431 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.888+0000 7f89b7fff640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89c00b3af0 0x7f89c01599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.888+0000 7f89bf7fe640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89c015b530 0x7f89c00b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.889+0000 7f89beffd640 1 -- 172.21.15.97:0/4084436431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89c00b3af0 msgr2=0x7f89c01599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.889+0000 7f89beffd640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89c00b3af0 0x7f89c01599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.889+0000 7f89beffd640 1 -- 172.21.15.97:0/4084436431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89c015b530 msgr2=0x7f89c00b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.889+0000 7f89beffd640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89c015b530 0x7f89c00b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.889+0000 7f89beffd640 1 -- 172.21.15.97:0/4084436431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c00afc70 con 0x7f89c006ca60 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.889+0000 7f89beffd640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c006ca60 0x7f89c0149bb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89b000c460 tx=0x7f89b000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.889+0000 7f89b77fe640 1 -- 172.21.15.97:0/4084436431 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0017070 con 0x7f89c006ca60 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.889+0000 7f89b77fe640 1 -- 172.21.15.97:0/4084436431 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b00052f0 con 0x7f89c006ca60 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.889+0000 7f89c531c640 1 -- 172.21.15.97:0/4084436431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c00aff50 con 0x7f89c006ca60 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.889+0000 7f89b77fe640 1 -- 172.21.15.97:0/4084436431 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0003830 con 0x7f89c006ca60 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.889+0000 7f89c531c640 1 -- 172.21.15.97:0/4084436431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c01558b0 con 0x7f89c006ca60 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.890+0000 7f89c531c640 1 -- 172.21.15.97:0/4084436431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c0068d50 con 0x7f89c006ca60 2024-03-29T04:07:43.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.890+0000 7f89b77fe640 1 -- 172.21.15.97:0/4084436431 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f89b001a070 con 0x7f89c006ca60 2024-03-29T04:07:43.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.891+0000 7f89b77fe640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f89a006c530 0x7f89a006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:43.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.891+0000 7f89b77fe640 1 -- 172.21.15.97:0/4084436431 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f89b008a9b0 con 0x7f89c006ca60 2024-03-29T04:07:43.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.891+0000 7f89b7fff640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f89a006c530 0x7f89a006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:43.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.891+0000 7f89b7fff640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f89a006c530 0x7f89a006e9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f89a8036000 tx=0x7f89a8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:43.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:43.895+0000 7f89b77fe640 1 -- 172.21.15.97:0/4084436431 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f89b0058a10 con 0x7f89c006ca60 2024-03-29T04:07:44.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.072+0000 7f89c531c640 1 -- 172.21.15.97:0/4084436431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89c0155280 con 0x7f89c006ca60 2024-03-29T04:07:44.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.073+0000 7f89b77fe640 1 -- 172.21.15.97:0/4084436431 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f89b005cd10 con 0x7f89c006ca60 2024-03-29T04:07:44.053 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:44.054 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:44.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.075+0000 7f89c531c640 1 -- 172.21.15.97:0/4084436431 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f89a006c530 msgr2=0x7f89a006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.075+0000 7f89c531c640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f89a006c530 0x7f89a006e9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f89a8036000 tx=0x7f89a8008030 comp rx=0 tx=0).stop 2024-03-29T04:07:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.076+0000 7f89c531c640 1 -- 172.21.15.97:0/4084436431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c006ca60 msgr2=0x7f89c0149bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.076+0000 7f89c531c640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c006ca60 0x7f89c0149bb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89b000c460 tx=0x7f89b000c920 comp rx=0 tx=0).stop 2024-03-29T04:07:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.076+0000 7f89c531c640 1 -- 172.21.15.97:0/4084436431 shutdown_connections 2024-03-29T04:07:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.076+0000 7f89c531c640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f89a006c530 0x7f89a006e9f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.076+0000 7f89c531c640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89c015b530 0x7f89c00b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.076+0000 7f89c531c640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89c00b3af0 0x7f89c01599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.076+0000 7f89c531c640 1 --2- 172.21.15.97:0/4084436431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89c006ca60 0x7f89c0149bb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.076+0000 7f89c531c640 1 -- 172.21.15.97:0/4084436431 >> 172.21.15.97:0/4084436431 conn(0x7f89c0072050 msgr2=0x7f89c00cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.076+0000 7f89c531c640 1 -- 172.21.15.97:0/4084436431 shutdown_connections 2024-03-29T04:07:44.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.076+0000 7f89c531c640 1 -- 172.21.15.97:0/4084436431 wait complete. 2024-03-29T04:07:44.066 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T04:07:44.067 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:07:44.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.186+0000 7f7b65590640 1 Processor -- start 2024-03-29T04:07:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.187+0000 7f7b65590640 1 -- start start 2024-03-29T04:07:44.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.187+0000 7f7b65590640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b6015e9b0 0x7f7b600d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.187+0000 7f7b65590640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b600d4f30 0x7f7b60162f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.187+0000 7f7b65590640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b600d2bf0 0x7f7b600d3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.187+0000 7f7b65590640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f7b601634b0 con 0x7f7b6015e9b0 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.187+0000 7f7b65590640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f7b600cfcf0 con 0x7f7b600d4f30 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.187+0000 7f7b65590640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7b600d54f0 con 0x7f7b600d2bf0 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.187+0000 7f7b5effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b6015e9b0 0x7f7b600d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.187+0000 7f7b5effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b6015e9b0 0x7f7b600d0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38682/0 (socket says 172.21.15.97:38682) 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.187+0000 7f7b5effd640 1 -- 172.21.15.97:0/442739069 learned_addr learned my addr 172.21.15.97:0/442739069 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.188+0000 7f7b567fc640 1 --2- 172.21.15.97:0/442739069 >> v2:172.21.15.154:3300/0 conn(0x7f7b600d4f30 0x7f7b60162f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.188+0000 7f7b5f7fe640 1 --2- 172.21.15.97:0/442739069 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b600d2bf0 0x7f7b600d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.188+0000 7f7b5effd640 1 -- 172.21.15.97:0/442739069 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b600d2bf0 msgr2=0x7f7b600d3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.188+0000 7f7b5effd640 1 --2- 172.21.15.97:0/442739069 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b600d2bf0 0x7f7b600d3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.188+0000 7f7b5effd640 1 -- 172.21.15.97:0/442739069 >> v2:172.21.15.154:3300/0 conn(0x7f7b600d4f30 msgr2=0x7f7b60162f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.188+0000 7f7b5effd640 1 --2- 172.21.15.97:0/442739069 >> v2:172.21.15.154:3300/0 conn(0x7f7b600d4f30 0x7f7b60162f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.188+0000 7f7b5effd640 1 -- 172.21.15.97:0/442739069 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b600cf3c0 con 0x7f7b6015e9b0 2024-03-29T04:07:44.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.188+0000 7f7b5effd640 1 --2- 172.21.15.97:0/442739069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b6015e9b0 0x7f7b600d0180 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7b600d2980 tx=0x7f7b5002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c562621c66aca8c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:44.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.189+0000 7f7b5e7fc640 1 -- 172.21.15.97:0/442739069 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50035020 con 0x7f7b6015e9b0 2024-03-29T04:07:44.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.189+0000 7f7b5e7fc640 1 -- 172.21.15.97:0/442739069 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b50031870 con 0x7f7b6015e9b0 2024-03-29T04:07:44.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.189+0000 7f7b5e7fc640 1 -- 172.21.15.97:0/442739069 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50031c40 con 0x7f7b6015e9b0 2024-03-29T04:07:44.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.189+0000 7f7b65590640 1 -- 172.21.15.97:0/442739069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b6015e9b0 msgr2=0x7f7b600d0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.189+0000 7f7b65590640 1 --2- 172.21.15.97:0/442739069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b6015e9b0 0x7f7b600d0180 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7b600d2980 tx=0x7f7b5002b910 comp rx=0 tx=0).stop 2024-03-29T04:07:44.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.189+0000 7f7b65590640 1 -- 172.21.15.97:0/442739069 shutdown_connections 2024-03-29T04:07:44.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.189+0000 7f7b65590640 1 --2- 172.21.15.97:0/442739069 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b600d2bf0 0x7f7b600d3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.189+0000 7f7b65590640 1 --2- 172.21.15.97:0/442739069 >> v2:172.21.15.154:3300/0 conn(0x7f7b600d4f30 0x7f7b60162f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.189+0000 7f7b65590640 1 --2- 172.21.15.97:0/442739069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b6015e9b0 0x7f7b600d0180 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.189+0000 7f7b65590640 1 -- 172.21.15.97:0/442739069 >> 172.21.15.97:0/442739069 conn(0x7f7b6008bfc0 msgr2=0x7f7b600d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:44.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.189+0000 7f7b65590640 1 -- 172.21.15.97:0/442739069 shutdown_connections 2024-03-29T04:07:44.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.189+0000 7f7b65590640 1 -- 172.21.15.97:0/442739069 wait complete. 2024-03-29T04:07:44.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.190+0000 7f7b65590640 1 Processor -- start 2024-03-29T04:07:44.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.191+0000 7f7b65590640 1 -- start start 2024-03-29T04:07:44.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.191+0000 7f7b65590640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b600d2bf0 0x7f7b6014f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.191+0000 7f7b65590640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b600d4f30 0x7f7b600ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.191+0000 7f7b65590640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b6015e9b0 0x7f7b600cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.191+0000 7f7b65590640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6015ed90 con 0x7f7b6015e9b0 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.191+0000 7f7b65590640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7b600cf980 con 0x7f7b600d2bf0 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.191+0000 7f7b65590640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6006c760 con 0x7f7b600d4f30 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.191+0000 7f7b5f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b6015e9b0 0x7f7b600cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.191+0000 7f7b5f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b6015e9b0 0x7f7b600cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38690/0 (socket says 172.21.15.97:38690) 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.191+0000 7f7b5f7fe640 1 -- 172.21.15.97:0/2795163727 learned_addr learned my addr 172.21.15.97:0/2795163727 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.191+0000 7f7b5effd640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b600d2bf0 0x7f7b6014f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.191+0000 7f7b567fc640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b600d4f30 0x7f7b600ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.192+0000 7f7b5f7fe640 1 -- 172.21.15.97:0/2795163727 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b600d4f30 msgr2=0x7f7b600ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.192+0000 7f7b5f7fe640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b600d4f30 0x7f7b600ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.192+0000 7f7b5f7fe640 1 -- 172.21.15.97:0/2795163727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b600d2bf0 msgr2=0x7f7b6014f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.192+0000 7f7b5f7fe640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b600d2bf0 0x7f7b6014f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.192+0000 7f7b5f7fe640 1 -- 172.21.15.97:0/2795163727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b600ca650 con 0x7f7b6015e9b0 2024-03-29T04:07:44.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.192+0000 7f7b5f7fe640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b6015e9b0 0x7f7b600cb790 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800eed0 tx=0x7f7b5800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:44.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.192+0000 7f7b57fff640 1 -- 172.21.15.97:0/2795163727 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58018020 con 0x7f7b6015e9b0 2024-03-29T04:07:44.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.192+0000 7f7b57fff640 1 -- 172.21.15.97:0/2795163727 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b58014400 con 0x7f7b6015e9b0 2024-03-29T04:07:44.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.192+0000 7f7b57fff640 1 -- 172.21.15.97:0/2795163727 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b580147f0 con 0x7f7b6015e9b0 2024-03-29T04:07:44.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.192+0000 7f7b65590640 1 -- 172.21.15.97:0/2795163727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b600ca920 con 0x7f7b6015e9b0 2024-03-29T04:07:44.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.192+0000 7f7b65590640 1 -- 172.21.15.97:0/2795163727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b600caee0 con 0x7f7b6015e9b0 2024-03-29T04:07:44.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.193+0000 7f7b65590640 1 -- 172.21.15.97:0/2795163727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b60068640 con 0x7f7b6015e9b0 2024-03-29T04:07:44.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.193+0000 7f7b57fff640 1 -- 172.21.15.97:0/2795163727 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7b58005d10 con 0x7f7b6015e9b0 2024-03-29T04:07:44.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.194+0000 7f7b57fff640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7b4006c530 0x7f7b4006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.194+0000 7f7b5effd640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7b4006c530 0x7f7b4006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.194+0000 7f7b57fff640 1 -- 172.21.15.97:0/2795163727 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7b5808bba0 con 0x7f7b6015e9b0 2024-03-29T04:07:44.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.194+0000 7f7b5effd640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7b4006c530 0x7f7b4006e9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7b50036000 tx=0x7f7b5003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:44.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.198+0000 7f7b57fff640 1 -- 172.21.15.97:0/2795163727 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7b58022070 con 0x7f7b6015e9b0 2024-03-29T04:07:44.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.371+0000 7f7b65590640 1 -- 172.21.15.97:0/2795163727 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b600d1d00 con 0x7f7b4006c530 2024-03-29T04:07:44.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.377+0000 7f7b57fff640 1 -- 172.21.15.97:0/2795163727 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237079 (secure 0 0 0) 0x7f7b600d1d00 con 0x7f7b4006c530 2024-03-29T04:07:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:44.363 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.380+0000 7f7b65590640 1 -- 172.21.15.97:0/2795163727 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7b4006c530 msgr2=0x7f7b4006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.380+0000 7f7b65590640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7b4006c530 0x7f7b4006e9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7b50036000 tx=0x7f7b5003c040 comp rx=0 tx=0).stop 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.380+0000 7f7b65590640 1 -- 172.21.15.97:0/2795163727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b6015e9b0 msgr2=0x7f7b600cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.380+0000 7f7b65590640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b6015e9b0 0x7f7b600cb790 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800eed0 tx=0x7f7b5800b480 comp rx=0 tx=0).stop 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.380+0000 7f7b65590640 1 -- 172.21.15.97:0/2795163727 shutdown_connections 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.380+0000 7f7b65590640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7b4006c530 0x7f7b4006e9f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.380+0000 7f7b65590640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b6015e9b0 0x7f7b600cb790 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.380+0000 7f7b65590640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7b600d4f30 0x7f7b600ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.380+0000 7f7b65590640 1 --2- 172.21.15.97:0/2795163727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b600d2bf0 0x7f7b6014f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.380+0000 7f7b65590640 1 -- 172.21.15.97:0/2795163727 >> 172.21.15.97:0/2795163727 conn(0x7f7b6008bfc0 msgr2=0x7f7b60160c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.380+0000 7f7b65590640 1 -- 172.21.15.97:0/2795163727 shutdown_connections 2024-03-29T04:07:44.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.380+0000 7f7b65590640 1 -- 172.21.15.97:0/2795163727 wait complete. 2024-03-29T04:07:44.371 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-29T04:07:42.638825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219176,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755544,"statfs":{"total":773094113280,"available":772869677056,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000619"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-29T04:07:32.273535+0000","last_change":"2024-03-29T04:07:29.856799+0000","last_active":"2024-03-29T04:07:32.273535+0000","last_peered":"2024-03-29T04:07:32.273535+0000","last_clean":"2024-03-29T04:07:32.273535+0000","last_became_active":"2024-03-29T04:07:29.855873+0000","last_became_peered":"2024-03-29T04:07:29.855873+0000","last_unstale":"2024-03-29T04:07:32.273535+0000","last_undegraded":"2024-03-29T04:07:32.273535+0000","last_fullsized":"2024-03-29T04:07:32.273535+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_clean_scrub_stamp":"2024-03-29T04:07:28.661516+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:25:15.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:40.403565+0000","last_change":"2024-03-29T04:07:40.403565+0000","last_active":"2024-03-29T04:07:40.403565+0000","last_peered":"2024-03-29T04:07:40.403565+0000","last_clean":"2024-03-29T04:07:40.403565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:40.403565+0000","last_undegraded":"2024-03-29T04:07:40.403565+0000","last_fullsized":"2024-03-29T04:07:40.403565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:07:39.398702+0000","last_change":"2024-03-29T04:07:39.398702+0000","last_active":"2024-03-29T04:07:39.398702+0000","last_peered":"2024-03-29T04:07:39.398702+0000","last_clean":"2024-03-29T04:07:39.398702+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:07:39.398702+0000","last_undegraded":"2024-03-29T04:07:39.398702+0000","last_fullsized":"2024-03-29T04:07:39.398702+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28376,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343464,"statfs":{"total":96636764160,"available":96607707136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:07:44.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:07:44.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.532+0000 7ff5c85d4640 1 Processor -- start 2024-03-29T04:07:44.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.532+0000 7ff5c85d4640 1 -- start start 2024-03-29T04:07:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.532+0000 7ff5c85d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5c006f3f0 0x7ff5c0072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.533+0000 7ff5c85d4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c0071ae0 con 0x7ff5c015b370 2024-03-29T04:07:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.533+0000 7ff5c85d4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c0071dc0 con 0x7ff5c00730c0 2024-03-29T04:07:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.533+0000 7ff5c85d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c00b39d0 con 0x7ff5c006f3f0 2024-03-29T04:07:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.533+0000 7ff5c6370640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff5c015b370 0x7ff5c00b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50340/0 (socket says 172.21.15.97:50340) 2024-03-29T04:07:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.533+0000 7ff5c6370640 1 -- 172.21.15.97:0/887938512 learned_addr learned my addr 172.21.15.97:0/887938512 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.533+0000 7ff5c5b6f640 1 --2- 172.21.15.97:0/887938512 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5c006f3f0 0x7ff5c0072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.533+0000 7ff5c536e640 1 -- 172.21.15.97:0/887938512 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1484085488 0 0) 0x7ff5c0071ae0 con 0x7ff5c015b370 2024-03-29T04:07:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.533+0000 7ff5c536e640 1 -- 172.21.15.97:0/887938512 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5ac003520 con 0x7ff5c015b370 2024-03-29T04:07:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.533+0000 7ff5c536e640 1 -- 172.21.15.97:0/887938512 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4163293391 0 0) 0x7ff5c0071dc0 con 0x7ff5c00730c0 2024-03-29T04:07:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.533+0000 7ff5c536e640 1 -- 172.21.15.97:0/887938512 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5c0071ae0 con 0x7ff5c00730c0 2024-03-29T04:07:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.533+0000 7ff5c5b6f640 1 -- 172.21.15.97:0/887938512 >> v1:172.21.15.154:6789/0 conn(0x7ff5c00730c0 legacy=0x7ff5c015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.533+0000 7ff5c5b6f640 1 -- 172.21.15.97:0/887938512 >> v1:172.21.15.97:6789/0 conn(0x7ff5c015b370 legacy=0x7ff5c00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.533+0000 7ff5c5b6f640 1 -- 172.21.15.97:0/887938512 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c00720a0 con 0x7ff5c006f3f0 2024-03-29T04:07:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.534+0000 7ff5c5b6f640 1 --2- 172.21.15.97:0/887938512 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5c006f3f0 0x7ff5c0072b80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40097d0 tx=0x7ff5b402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8cb8a7312b1a8fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:44.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.534+0000 7ff5c536e640 1 -- 172.21.15.97:0/887938512 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4035020 con 0x7ff5c006f3f0 2024-03-29T04:07:44.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.534+0000 7ff5c536e640 1 -- 172.21.15.97:0/887938512 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b4031890 con 0x7ff5c006f3f0 2024-03-29T04:07:44.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.534+0000 7ff5c536e640 1 -- 172.21.15.97:0/887938512 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4031c40 con 0x7ff5c006f3f0 2024-03-29T04:07:44.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.534+0000 7ff5c85d4640 1 -- 172.21.15.97:0/887938512 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5c006f3f0 msgr2=0x7ff5c0072b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.534+0000 7ff5c85d4640 1 --2- 172.21.15.97:0/887938512 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5c006f3f0 0x7ff5c0072b80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40097d0 tx=0x7ff5b402b930 comp rx=0 tx=0).stop 2024-03-29T04:07:44.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.535+0000 7ff5c85d4640 1 -- 172.21.15.97:0/887938512 shutdown_connections 2024-03-29T04:07:44.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.535+0000 7ff5c85d4640 1 --2- 172.21.15.97:0/887938512 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5c006f3f0 0x7ff5c0072b80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.535+0000 7ff5c85d4640 1 -- 172.21.15.97:0/887938512 >> 172.21.15.97:0/887938512 conn(0x7ff5c006b510 msgr2=0x7ff5c006b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:44.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.535+0000 7ff5c85d4640 1 -- 172.21.15.97:0/887938512 shutdown_connections 2024-03-29T04:07:44.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.535+0000 7ff5c85d4640 1 -- 172.21.15.97:0/887938512 wait complete. 2024-03-29T04:07:44.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.536+0000 7ff5c85d4640 1 Processor -- start 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.536+0000 7ff5c85d4640 1 -- start start 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.536+0000 7ff5c85d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5c006f3f0 0x7ff5c014dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.536+0000 7ff5c85d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5c00730c0 0x7ff5c015bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.536+0000 7ff5c85d4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c015b370 0x7ff5c015c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.536+0000 7ff5c85d4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c0073550 con 0x7ff5c015b370 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.536+0000 7ff5c85d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c0072660 con 0x7ff5c00730c0 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.536+0000 7ff5c85d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c015b750 con 0x7ff5c006f3f0 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.536+0000 7ff5c6b71640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c015b370 0x7ff5c015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.536+0000 7ff5c6370640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5c006f3f0 0x7ff5c014dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5c6b71640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c015b370 0x7ff5c015c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38692/0 (socket says 172.21.15.97:38692) 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5c6b71640 1 -- 172.21.15.97:0/4065369421 learned_addr learned my addr 172.21.15.97:0/4065369421 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5c5b6f640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5c00730c0 0x7ff5c015bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5c6b71640 1 -- 172.21.15.97:0/4065369421 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5c006f3f0 msgr2=0x7ff5c014dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5c6b71640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5c006f3f0 0x7ff5c014dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5c6b71640 1 -- 172.21.15.97:0/4065369421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5c00730c0 msgr2=0x7ff5c015bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5c6b71640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5c00730c0 0x7ff5c015bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5c6b71640 1 -- 172.21.15.97:0/4065369421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c014e4f0 con 0x7ff5c015b370 2024-03-29T04:07:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5c6370640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5c006f3f0 0x7ff5c014dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5c6b71640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c015b370 0x7ff5c015c260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800c790 tx=0x7ff5b800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5b2ffd640 1 -- 172.21.15.97:0/4065369421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8003b70 con 0x7ff5c015b370 2024-03-29T04:07:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5b2ffd640 1 -- 172.21.15.97:0/4065369421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b8003d10 con 0x7ff5c015b370 2024-03-29T04:07:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5c85d4640 1 -- 172.21.15.97:0/4065369421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c0152420 con 0x7ff5c015b370 2024-03-29T04:07:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5b2ffd640 1 -- 172.21.15.97:0/4065369421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b800aa50 con 0x7ff5c015b370 2024-03-29T04:07:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.537+0000 7ff5c85d4640 1 -- 172.21.15.97:0/4065369421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c014d4d0 con 0x7ff5c015b370 2024-03-29T04:07:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.538+0000 7ff5c85d4640 1 -- 172.21.15.97:0/4065369421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c00b06f0 con 0x7ff5c015b370 2024-03-29T04:07:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.538+0000 7ff5b2ffd640 1 -- 172.21.15.97:0/4065369421 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff5b800abf0 con 0x7ff5c015b370 2024-03-29T04:07:44.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.539+0000 7ff5b2ffd640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff5a406c530 0x7ff5a406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.539+0000 7ff5b2ffd640 1 -- 172.21.15.97:0/4065369421 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff5b808b510 con 0x7ff5c015b370 2024-03-29T04:07:44.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.539+0000 7ff5c6370640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff5a406c530 0x7ff5a406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.539+0000 7ff5c6370640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff5a406c530 0x7ff5a406e9f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc000f60 tx=0x7ff5bc007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:44.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.543+0000 7ff5b2ffd640 1 -- 172.21.15.97:0/4065369421 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff5c00b06f0 con 0x7ff5c015b370 2024-03-29T04:07:44.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.716+0000 7ff5c85d4640 1 -- 172.21.15.97:0/4065369421 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5c014d010 con 0x7ff5a406c530 2024-03-29T04:07:44.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.722+0000 7ff5b2ffd640 1 -- 172.21.15.97:0/4065369421 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251899 (secure 0 0 0) 0x7ff5bc007690 con 0x7ff5a406c530 2024-03-29T04:07:44.704 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:44.705 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-29T04:07:44.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.724+0000 7ff5c85d4640 1 -- 172.21.15.97:0/4065369421 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff5a406c530 msgr2=0x7ff5a406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.724+0000 7ff5c85d4640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff5a406c530 0x7ff5a406e9f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc000f60 tx=0x7ff5bc007d50 comp rx=0 tx=0).stop 2024-03-29T04:07:44.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.724+0000 7ff5c85d4640 1 -- 172.21.15.97:0/4065369421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c015b370 msgr2=0x7ff5c015c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.724+0000 7ff5c85d4640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c015b370 0x7ff5c015c260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800c790 tx=0x7ff5b800cc50 comp rx=0 tx=0).stop 2024-03-29T04:07:44.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.724+0000 7ff5c85d4640 1 -- 172.21.15.97:0/4065369421 shutdown_connections 2024-03-29T04:07:44.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.724+0000 7ff5c85d4640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff5a406c530 0x7ff5a406e9f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.724+0000 7ff5c85d4640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c015b370 0x7ff5c015c260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.724+0000 7ff5c85d4640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5c00730c0 0x7ff5c015bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.724+0000 7ff5c85d4640 1 --2- 172.21.15.97:0/4065369421 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5c006f3f0 0x7ff5c014dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.724+0000 7ff5c85d4640 1 -- 172.21.15.97:0/4065369421 >> 172.21.15.97:0/4065369421 conn(0x7ff5c006b510 msgr2=0x7ff5c0157740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:44.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.725+0000 7ff5c85d4640 1 -- 172.21.15.97:0/4065369421 shutdown_connections 2024-03-29T04:07:44.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.725+0000 7ff5c85d4640 1 -- 172.21.15.97:0/4065369421 wait complete. 2024-03-29T04:07:44.715 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-29T04:07:44.639096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218900,"kb_used_data":2728,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755820,"statfs":{"total":773094113280,"available":772869959680,"internally_reserved":0,"allocated":2793472,"data_stored":1702628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.235527"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.426834+0000","last_change":"2024-03-29T04:07:41.422348+0000","last_active":"2024-03-29T04:07:41.426834+0000","last_peered":"2024-03-29T04:07:41.426834+0000","last_clean":"2024-03-29T04:07:41.426834+0000","last_became_active":"2024-03-29T04:07:41.422010+0000","last_became_peered":"2024-03-29T04:07:41.422010+0000","last_unstale":"2024-03-29T04:07:41.426834+0000","last_undegraded":"2024-03-29T04:07:41.426834+0000","last_fullsized":"2024-03-29T04:07:41.426834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:19.861306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413460+0000","last_change":"2024-03-29T04:07:41.426583+0000","last_active":"2024-03-29T04:07:42.413460+0000","last_peered":"2024-03-29T04:07:42.413460+0000","last_clean":"2024-03-29T04:07:42.413460+0000","last_became_active":"2024-03-29T04:07:41.426321+0000","last_became_peered":"2024-03-29T04:07:41.426321+0000","last_unstale":"2024-03-29T04:07:42.413460+0000","last_undegraded":"2024-03-29T04:07:42.413460+0000","last_fullsized":"2024-03-29T04:07:42.413460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:28:53.046219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.439634+0000","last_change":"2024-03-29T04:07:41.430842+0000","last_active":"2024-03-29T04:07:41.439634+0000","last_peered":"2024-03-29T04:07:41.439634+0000","last_clean":"2024-03-29T04:07:41.439634+0000","last_became_active":"2024-03-29T04:07:41.430402+0000","last_became_peered":"2024-03-29T04:07:41.430402+0000","last_unstale":"2024-03-29T04:07:41.439634+0000","last_undegraded":"2024-03-29T04:07:41.439634+0000","last_fullsized":"2024-03-29T04:07:41.439634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:35.964938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.444601+0000","last_change":"2024-03-29T04:07:41.436930+0000","last_active":"2024-03-29T04:07:41.444601+0000","last_peered":"2024-03-29T04:07:41.444601+0000","last_clean":"2024-03-29T04:07:41.444601+0000","last_became_active":"2024-03-29T04:07:41.436434+0000","last_became_peered":"2024-03-29T04:07:41.436434+0000","last_unstale":"2024-03-29T04:07:41.444601+0000","last_undegraded":"2024-03-29T04:07:41.444601+0000","last_fullsized":"2024-03-29T04:07:41.444601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:31:27.159107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.435876+0000","last_change":"2024-03-29T04:07:41.428211+0000","last_active":"2024-03-29T04:07:41.435876+0000","last_peered":"2024-03-29T04:07:41.435876+0000","last_clean":"2024-03-29T04:07:41.435876+0000","last_became_active":"2024-03-29T04:07:41.427906+0000","last_became_peered":"2024-03-29T04:07:41.427906+0000","last_unstale":"2024-03-29T04:07:41.435876+0000","last_undegraded":"2024-03-29T04:07:41.435876+0000","last_fullsized":"2024-03-29T04:07:41.435876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:57.438333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.444777+0000","last_change":"2024-03-29T04:07:41.438804+0000","last_active":"2024-03-29T04:07:41.444777+0000","last_peered":"2024-03-29T04:07:41.444777+0000","last_clean":"2024-03-29T04:07:41.444777+0000","last_became_active":"2024-03-29T04:07:41.438533+0000","last_became_peered":"2024-03-29T04:07:41.438533+0000","last_unstale":"2024-03-29T04:07:41.444777+0000","last_undegraded":"2024-03-29T04:07:41.444777+0000","last_fullsized":"2024-03-29T04:07:41.444777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:57.438333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.435476+0000","last_change":"2024-03-29T04:07:41.426635+0000","last_active":"2024-03-29T04:07:41.435476+0000","last_peered":"2024-03-29T04:07:41.435476+0000","last_clean":"2024-03-29T04:07:41.435476+0000","last_became_active":"2024-03-29T04:07:41.426301+0000","last_became_peered":"2024-03-29T04:07:41.426301+0000","last_unstale":"2024-03-29T04:07:41.435476+0000","last_undegraded":"2024-03-29T04:07:41.435476+0000","last_fullsized":"2024-03-29T04:07:41.435476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:10:59.508554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.439857+0000","last_change":"2024-03-29T04:07:41.436075+0000","last_active":"2024-03-29T04:07:41.439857+0000","last_peered":"2024-03-29T04:07:41.439857+0000","last_clean":"2024-03-29T04:07:41.439857+0000","last_became_active":"2024-03-29T04:07:41.435833+0000","last_became_peered":"2024-03-29T04:07:41.435833+0000","last_unstale":"2024-03-29T04:07:41.439857+0000","last_undegraded":"2024-03-29T04:07:41.439857+0000","last_fullsized":"2024-03-29T04:07:41.439857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:19.861306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.442570+0000","last_change":"2024-03-29T04:07:41.441371+0000","last_active":"2024-03-29T04:07:41.442570+0000","last_peered":"2024-03-29T04:07:41.442570+0000","last_clean":"2024-03-29T04:07:41.442570+0000","last_became_active":"2024-03-29T04:07:41.441008+0000","last_became_peered":"2024-03-29T04:07:41.441008+0000","last_unstale":"2024-03-29T04:07:41.442570+0000","last_undegraded":"2024-03-29T04:07:41.442570+0000","last_fullsized":"2024-03-29T04:07:41.442570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:32:16.919913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417268+0000","last_change":"2024-03-29T04:07:41.436960+0000","last_active":"2024-03-29T04:07:42.417268+0000","last_peered":"2024-03-29T04:07:42.417268+0000","last_clean":"2024-03-29T04:07:42.417268+0000","last_became_active":"2024-03-29T04:07:41.436652+0000","last_became_peered":"2024-03-29T04:07:41.436652+0000","last_unstale":"2024-03-29T04:07:42.417268+0000","last_undegraded":"2024-03-29T04:07:42.417268+0000","last_fullsized":"2024-03-29T04:07:42.417268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:23:32.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417196+0000","last_change":"2024-03-29T04:07:41.437571+0000","last_active":"2024-03-29T04:07:42.417196+0000","last_peered":"2024-03-29T04:07:42.417196+0000","last_clean":"2024-03-29T04:07:42.417196+0000","last_became_active":"2024-03-29T04:07:41.437307+0000","last_became_peered":"2024-03-29T04:07:41.437307+0000","last_unstale":"2024-03-29T04:07:42.417196+0000","last_undegraded":"2024-03-29T04:07:42.417196+0000","last_fullsized":"2024-03-29T04:07:42.417196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:30:59.143986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417226+0000","last_change":"2024-03-29T04:07:41.429087+0000","last_active":"2024-03-29T04:07:42.417226+0000","last_peered":"2024-03-29T04:07:42.417226+0000","last_clean":"2024-03-29T04:07:42.417226+0000","last_became_active":"2024-03-29T04:07:41.427764+0000","last_became_peered":"2024-03-29T04:07:41.427764+0000","last_unstale":"2024-03-29T04:07:42.417226+0000","last_undegraded":"2024-03-29T04:07:42.417226+0000","last_fullsized":"2024-03-29T04:07:42.417226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:12.561151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.432312+0000","last_change":"2024-03-29T04:07:41.427842+0000","last_active":"2024-03-29T04:07:41.432312+0000","last_peered":"2024-03-29T04:07:41.432312+0000","last_clean":"2024-03-29T04:07:41.432312+0000","last_became_active":"2024-03-29T04:07:41.427481+0000","last_became_peered":"2024-03-29T04:07:41.427481+0000","last_unstale":"2024-03-29T04:07:41.432312+0000","last_undegraded":"2024-03-29T04:07:41.432312+0000","last_fullsized":"2024-03-29T04:07:41.432312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:42.838588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.430487+0000","last_change":"2024-03-29T04:07:41.425389+0000","last_active":"2024-03-29T04:07:41.430487+0000","last_peered":"2024-03-29T04:07:41.430487+0000","last_clean":"2024-03-29T04:07:41.430487+0000","last_became_active":"2024-03-29T04:07:41.425016+0000","last_became_peered":"2024-03-29T04:07:41.425016+0000","last_unstale":"2024-03-29T04:07:41.430487+0000","last_undegraded":"2024-03-29T04:07:41.430487+0000","last_fullsized":"2024-03-29T04:07:41.430487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:41.466887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413581+0000","last_change":"2024-03-29T04:07:41.428282+0000","last_active":"2024-03-29T04:07:42.413581+0000","last_peered":"2024-03-29T04:07:42.413581+0000","last_clean":"2024-03-29T04:07:42.413581+0000","last_became_active":"2024-03-29T04:07:41.428042+0000","last_became_peered":"2024-03-29T04:07:41.428042+0000","last_unstale":"2024-03-29T04:07:42.413581+0000","last_undegraded":"2024-03-29T04:07:42.413581+0000","last_fullsized":"2024-03-29T04:07:42.413581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:23:32.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.441559+0000","last_change":"2024-03-29T04:07:41.434962+0000","last_active":"2024-03-29T04:07:41.441559+0000","last_peered":"2024-03-29T04:07:41.441559+0000","last_clean":"2024-03-29T04:07:41.441559+0000","last_became_active":"2024-03-29T04:07:41.434691+0000","last_became_peered":"2024-03-29T04:07:41.434691+0000","last_unstale":"2024-03-29T04:07:41.441559+0000","last_undegraded":"2024-03-29T04:07:41.441559+0000","last_fullsized":"2024-03-29T04:07:41.441559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:39:47.907501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.441483+0000","last_change":"2024-03-29T04:07:41.434716+0000","last_active":"2024-03-29T04:07:41.441483+0000","last_peered":"2024-03-29T04:07:41.441483+0000","last_clean":"2024-03-29T04:07:41.441483+0000","last_became_active":"2024-03-29T04:07:41.434372+0000","last_became_peered":"2024-03-29T04:07:41.434372+0000","last_unstale":"2024-03-29T04:07:41.441483+0000","last_undegraded":"2024-03-29T04:07:41.441483+0000","last_fullsized":"2024-03-29T04:07:41.441483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:38.089586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.432144+0000","last_change":"2024-03-29T04:07:41.427613+0000","last_active":"2024-03-29T04:07:41.432144+0000","last_peered":"2024-03-29T04:07:41.432144+0000","last_clean":"2024-03-29T04:07:41.432144+0000","last_became_active":"2024-03-29T04:07:41.427338+0000","last_became_peered":"2024-03-29T04:07:41.427338+0000","last_unstale":"2024-03-29T04:07:41.432144+0000","last_undegraded":"2024-03-29T04:07:41.432144+0000","last_fullsized":"2024-03-29T04:07:41.432144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:12.561151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.435897+0000","last_change":"2024-03-29T04:07:41.427942+0000","last_active":"2024-03-29T04:07:41.435897+0000","last_peered":"2024-03-29T04:07:41.435897+0000","last_clean":"2024-03-29T04:07:41.435897+0000","last_became_active":"2024-03-29T04:07:41.427667+0000","last_became_peered":"2024-03-29T04:07:41.427667+0000","last_unstale":"2024-03-29T04:07:41.435897+0000","last_undegraded":"2024-03-29T04:07:41.435897+0000","last_fullsized":"2024-03-29T04:07:41.435897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:31:27.159107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417255+0000","last_change":"2024-03-29T04:07:41.429646+0000","last_active":"2024-03-29T04:07:42.417255+0000","last_peered":"2024-03-29T04:07:42.417255+0000","last_clean":"2024-03-29T04:07:42.417255+0000","last_became_active":"2024-03-29T04:07:41.429376+0000","last_became_peered":"2024-03-29T04:07:41.429376+0000","last_unstale":"2024-03-29T04:07:42.417255+0000","last_undegraded":"2024-03-29T04:07:42.417255+0000","last_fullsized":"2024-03-29T04:07:42.417255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:35.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.430607+0000","last_change":"2024-03-29T04:07:41.425314+0000","last_active":"2024-03-29T04:07:41.430607+0000","last_peered":"2024-03-29T04:07:41.430607+0000","last_clean":"2024-03-29T04:07:41.430607+0000","last_became_active":"2024-03-29T04:07:41.424897+0000","last_became_peered":"2024-03-29T04:07:41.424897+0000","last_unstale":"2024-03-29T04:07:41.430607+0000","last_undegraded":"2024-03-29T04:07:41.430607+0000","last_fullsized":"2024-03-29T04:07:41.430607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:05.929698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.447293+0000","last_change":"2024-03-29T04:07:41.441556+0000","last_active":"2024-03-29T04:07:41.447293+0000","last_peered":"2024-03-29T04:07:41.447293+0000","last_clean":"2024-03-29T04:07:41.447293+0000","last_became_active":"2024-03-29T04:07:41.437251+0000","last_became_peered":"2024-03-29T04:07:41.437251+0000","last_unstale":"2024-03-29T04:07:41.447293+0000","last_undegraded":"2024-03-29T04:07:41.447293+0000","last_fullsized":"2024-03-29T04:07:41.447293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:28:53.046219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.439521+0000","last_change":"2024-03-29T04:07:41.435226+0000","last_active":"2024-03-29T04:07:41.439521+0000","last_peered":"2024-03-29T04:07:41.439521+0000","last_clean":"2024-03-29T04:07:41.439521+0000","last_became_active":"2024-03-29T04:07:41.434543+0000","last_became_peered":"2024-03-29T04:07:41.434543+0000","last_unstale":"2024-03-29T04:07:41.439521+0000","last_undegraded":"2024-03-29T04:07:41.439521+0000","last_fullsized":"2024-03-29T04:07:41.439521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:38.089586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.441535+0000","last_change":"2024-03-29T04:07:41.434893+0000","last_active":"2024-03-29T04:07:41.441535+0000","last_peered":"2024-03-29T04:07:41.441535+0000","last_clean":"2024-03-29T04:07:41.441535+0000","last_became_active":"2024-03-29T04:07:41.434578+0000","last_became_peered":"2024-03-29T04:07:41.434578+0000","last_unstale":"2024-03-29T04:07:41.441535+0000","last_undegraded":"2024-03-29T04:07:41.441535+0000","last_fullsized":"2024-03-29T04:07:41.441535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:50:51.620387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.439430+0000","last_change":"2024-03-29T04:07:41.434765+0000","last_active":"2024-03-29T04:07:41.439430+0000","last_peered":"2024-03-29T04:07:41.439430+0000","last_clean":"2024-03-29T04:07:41.439430+0000","last_became_active":"2024-03-29T04:07:41.434463+0000","last_became_peered":"2024-03-29T04:07:41.434463+0000","last_unstale":"2024-03-29T04:07:41.439430+0000","last_undegraded":"2024-03-29T04:07:41.439430+0000","last_fullsized":"2024-03-29T04:07:41.439430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:39:47.907501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417218+0000","last_change":"2024-03-29T04:07:41.424421+0000","last_active":"2024-03-29T04:07:42.417218+0000","last_peered":"2024-03-29T04:07:42.417218+0000","last_clean":"2024-03-29T04:07:42.417218+0000","last_became_active":"2024-03-29T04:07:41.424154+0000","last_became_peered":"2024-03-29T04:07:41.424154+0000","last_unstale":"2024-03-29T04:07:42.417218+0000","last_undegraded":"2024-03-29T04:07:42.417218+0000","last_fullsized":"2024-03-29T04:07:42.417218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:28:53.046219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413456+0000","last_change":"2024-03-29T04:07:41.429051+0000","last_active":"2024-03-29T04:07:42.413456+0000","last_peered":"2024-03-29T04:07:42.413456+0000","last_clean":"2024-03-29T04:07:42.413456+0000","last_became_active":"2024-03-29T04:07:41.428748+0000","last_became_peered":"2024-03-29T04:07:41.428748+0000","last_unstale":"2024-03-29T04:07:42.413456+0000","last_undegraded":"2024-03-29T04:07:42.413456+0000","last_fullsized":"2024-03-29T04:07:42.413456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:30:59.143986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.439551+0000","last_change":"2024-03-29T04:07:41.435463+0000","last_active":"2024-03-29T04:07:41.439551+0000","last_peered":"2024-03-29T04:07:41.439551+0000","last_clean":"2024-03-29T04:07:41.439551+0000","last_became_active":"2024-03-29T04:07:41.434902+0000","last_became_peered":"2024-03-29T04:07:41.434902+0000","last_unstale":"2024-03-29T04:07:41.439551+0000","last_undegraded":"2024-03-29T04:07:41.439551+0000","last_fullsized":"2024-03-29T04:07:41.439551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:36:03.632489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.435773+0000","last_change":"2024-03-29T04:07:41.428583+0000","last_active":"2024-03-29T04:07:41.435773+0000","last_peered":"2024-03-29T04:07:41.435773+0000","last_clean":"2024-03-29T04:07:41.435773+0000","last_became_active":"2024-03-29T04:07:41.428264+0000","last_became_peered":"2024-03-29T04:07:41.428264+0000","last_unstale":"2024-03-29T04:07:41.435773+0000","last_undegraded":"2024-03-29T04:07:41.435773+0000","last_fullsized":"2024-03-29T04:07:41.435773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:09.913867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.441706+0000","last_change":"2024-03-29T04:07:41.433555+0000","last_active":"2024-03-29T04:07:41.441706+0000","last_peered":"2024-03-29T04:07:41.441706+0000","last_clean":"2024-03-29T04:07:41.441706+0000","last_became_active":"2024-03-29T04:07:41.433253+0000","last_became_peered":"2024-03-29T04:07:41.433253+0000","last_unstale":"2024-03-29T04:07:41.441706+0000","last_undegraded":"2024-03-29T04:07:41.441706+0000","last_fullsized":"2024-03-29T04:07:41.441706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:45.683611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.436563+0000","last_change":"2024-03-29T04:07:41.433903+0000","last_active":"2024-03-29T04:07:41.436563+0000","last_peered":"2024-03-29T04:07:41.436563+0000","last_clean":"2024-03-29T04:07:41.436563+0000","last_became_active":"2024-03-29T04:07:41.433582+0000","last_became_peered":"2024-03-29T04:07:41.433582+0000","last_unstale":"2024-03-29T04:07:41.436563+0000","last_undegraded":"2024-03-29T04:07:41.436563+0000","last_fullsized":"2024-03-29T04:07:41.436563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:35.964938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417165+0000","last_change":"2024-03-29T04:07:41.426138+0000","last_active":"2024-03-29T04:07:42.417165+0000","last_peered":"2024-03-29T04:07:42.417165+0000","last_clean":"2024-03-29T04:07:42.417165+0000","last_became_active":"2024-03-29T04:07:41.425636+0000","last_became_peered":"2024-03-29T04:07:41.425636+0000","last_unstale":"2024-03-29T04:07:42.417165+0000","last_undegraded":"2024-03-29T04:07:42.417165+0000","last_fullsized":"2024-03-29T04:07:42.417165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:31:27.159107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417281+0000","last_change":"2024-03-29T04:07:41.434449+0000","last_active":"2024-03-29T04:07:42.417281+0000","last_peered":"2024-03-29T04:07:42.417281+0000","last_clean":"2024-03-29T04:07:42.417281+0000","last_became_active":"2024-03-29T04:07:41.434169+0000","last_became_peered":"2024-03-29T04:07:41.434169+0000","last_unstale":"2024-03-29T04:07:42.417281+0000","last_undegraded":"2024-03-29T04:07:42.417281+0000","last_fullsized":"2024-03-29T04:07:42.417281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:35.964938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.441382+0000","last_change":"2024-03-29T04:07:40.425200+0000","last_active":"2024-03-29T04:07:41.441382+0000","last_peered":"2024-03-29T04:07:41.441382+0000","last_clean":"2024-03-29T04:07:41.441382+0000","last_became_active":"2024-03-29T04:07:40.424061+0000","last_became_peered":"2024-03-29T04:07:40.424061+0000","last_unstale":"2024-03-29T04:07:41.441382+0000","last_undegraded":"2024-03-29T04:07:41.441382+0000","last_fullsized":"2024-03-29T04:07:41.441382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:04.924835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.439708+0000","last_change":"2024-03-29T04:07:41.432958+0000","last_active":"2024-03-29T04:07:41.439708+0000","last_peered":"2024-03-29T04:07:41.439708+0000","last_clean":"2024-03-29T04:07:41.439708+0000","last_became_active":"2024-03-29T04:07:41.432654+0000","last_became_peered":"2024-03-29T04:07:41.432654+0000","last_unstale":"2024-03-29T04:07:41.439708+0000","last_undegraded":"2024-03-29T04:07:41.439708+0000","last_fullsized":"2024-03-29T04:07:41.439708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:45.683611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.416646+0000","last_change":"2024-03-29T04:07:40.418823+0000","last_active":"2024-03-29T04:07:42.416646+0000","last_peered":"2024-03-29T04:07:42.416646+0000","last_clean":"2024-03-29T04:07:42.416646+0000","last_became_active":"2024-03-29T04:07:40.418542+0000","last_became_peered":"2024-03-29T04:07:40.418542+0000","last_unstale":"2024-03-29T04:07:42.416646+0000","last_undegraded":"2024-03-29T04:07:42.416646+0000","last_fullsized":"2024-03-29T04:07:42.416646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:04.924835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.417414+0000","last_change":"2024-03-29T04:07:40.430532+0000","last_active":"2024-03-29T04:07:41.417414+0000","last_peered":"2024-03-29T04:07:41.417414+0000","last_clean":"2024-03-29T04:07:41.417414+0000","last_became_active":"2024-03-29T04:07:40.430321+0000","last_became_peered":"2024-03-29T04:07:40.430321+0000","last_unstale":"2024-03-29T04:07:41.417414+0000","last_undegraded":"2024-03-29T04:07:41.417414+0000","last_fullsized":"2024-03-29T04:07:41.417414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:40:12.105363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.429904+0000","last_change":"2024-03-29T04:07:41.426187+0000","last_active":"2024-03-29T04:07:41.429904+0000","last_peered":"2024-03-29T04:07:41.429904+0000","last_clean":"2024-03-29T04:07:41.429904+0000","last_became_active":"2024-03-29T04:07:41.425926+0000","last_became_peered":"2024-03-29T04:07:41.425926+0000","last_unstale":"2024-03-29T04:07:41.429904+0000","last_undegraded":"2024-03-29T04:07:41.429904+0000","last_fullsized":"2024-03-29T04:07:41.429904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:12.561151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.424349+0000","last_change":"2024-03-29T04:07:40.430187+0000","last_active":"2024-03-29T04:07:41.424349+0000","last_peered":"2024-03-29T04:07:41.424349+0000","last_clean":"2024-03-29T04:07:41.424349+0000","last_became_active":"2024-03-29T04:07:40.429878+0000","last_became_peered":"2024-03-29T04:07:40.429878+0000","last_unstale":"2024-03-29T04:07:41.424349+0000","last_undegraded":"2024-03-29T04:07:41.424349+0000","last_fullsized":"2024-03-29T04:07:41.424349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413340+0000","last_change":"2024-03-29T04:07:41.427276+0000","last_active":"2024-03-29T04:07:42.413340+0000","last_peered":"2024-03-29T04:07:42.413340+0000","last_clean":"2024-03-29T04:07:42.413340+0000","last_became_active":"2024-03-29T04:07:41.427019+0000","last_became_peered":"2024-03-29T04:07:41.427019+0000","last_unstale":"2024-03-29T04:07:42.413340+0000","last_undegraded":"2024-03-29T04:07:42.413340+0000","last_fullsized":"2024-03-29T04:07:42.413340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:31:27.159107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.416916+0000","last_change":"2024-03-29T04:07:40.430476+0000","last_active":"2024-03-29T04:07:42.416916+0000","last_peered":"2024-03-29T04:07:42.416916+0000","last_clean":"2024-03-29T04:07:42.416916+0000","last_became_active":"2024-03-29T04:07:40.430169+0000","last_became_peered":"2024-03-29T04:07:40.430169+0000","last_unstale":"2024-03-29T04:07:42.416916+0000","last_undegraded":"2024-03-29T04:07:42.416916+0000","last_fullsized":"2024-03-29T04:07:42.416916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.444729+0000","last_change":"2024-03-29T04:07:41.439222+0000","last_active":"2024-03-29T04:07:41.444729+0000","last_peered":"2024-03-29T04:07:41.444729+0000","last_clean":"2024-03-29T04:07:41.444729+0000","last_became_active":"2024-03-29T04:07:41.437292+0000","last_became_peered":"2024-03-29T04:07:41.437292+0000","last_unstale":"2024-03-29T04:07:41.444729+0000","last_undegraded":"2024-03-29T04:07:41.444729+0000","last_fullsized":"2024-03-29T04:07:41.444729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:30:59.143986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.424889+0000","last_change":"2024-03-29T04:07:40.425014+0000","last_active":"2024-03-29T04:07:41.424889+0000","last_peered":"2024-03-29T04:07:41.424889+0000","last_clean":"2024-03-29T04:07:41.424889+0000","last_became_active":"2024-03-29T04:07:40.424254+0000","last_became_peered":"2024-03-29T04:07:40.424254+0000","last_unstale":"2024-03-29T04:07:41.424889+0000","last_undegraded":"2024-03-29T04:07:41.424889+0000","last_fullsized":"2024-03-29T04:07:41.424889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:08.909004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.442152+0000","last_change":"2024-03-29T04:07:41.436336+0000","last_active":"2024-03-29T04:07:41.442152+0000","last_peered":"2024-03-29T04:07:41.442152+0000","last_clean":"2024-03-29T04:07:41.442152+0000","last_became_active":"2024-03-29T04:07:41.436089+0000","last_became_peered":"2024-03-29T04:07:41.436089+0000","last_unstale":"2024-03-29T04:07:41.442152+0000","last_undegraded":"2024-03-29T04:07:41.442152+0000","last_fullsized":"2024-03-29T04:07:41.442152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:50:51.620387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413545+0000","last_change":"2024-03-29T04:07:40.424800+0000","last_active":"2024-03-29T04:07:42.413545+0000","last_peered":"2024-03-29T04:07:42.413545+0000","last_clean":"2024-03-29T04:07:42.413545+0000","last_became_active":"2024-03-29T04:07:40.424508+0000","last_became_peered":"2024-03-29T04:07:40.424508+0000","last_unstale":"2024-03-29T04:07:42.413545+0000","last_undegraded":"2024-03-29T04:07:42.413545+0000","last_fullsized":"2024-03-29T04:07:42.413545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:44.678749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417383+0000","last_change":"2024-03-29T04:07:41.429324+0000","last_active":"2024-03-29T04:07:42.417383+0000","last_peered":"2024-03-29T04:07:42.417383+0000","last_clean":"2024-03-29T04:07:42.417383+0000","last_became_active":"2024-03-29T04:07:41.428117+0000","last_became_peered":"2024-03-29T04:07:41.428117+0000","last_unstale":"2024-03-29T04:07:42.417383+0000","last_undegraded":"2024-03-29T04:07:42.417383+0000","last_fullsized":"2024-03-29T04:07:42.417383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:45.683611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417360+0000","last_change":"2024-03-29T04:07:40.428764+0000","last_active":"2024-03-29T04:07:42.417360+0000","last_peered":"2024-03-29T04:07:42.417360+0000","last_clean":"2024-03-29T04:07:42.417360+0000","last_became_active":"2024-03-29T04:07:40.428510+0000","last_became_peered":"2024-03-29T04:07:40.428510+0000","last_unstale":"2024-03-29T04:07:42.417360+0000","last_undegraded":"2024-03-29T04:07:42.417360+0000","last_fullsized":"2024-03-29T04:07:42.417360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:34.960075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.438800+0000","last_change":"2024-03-29T04:07:41.431774+0000","last_active":"2024-03-29T04:07:41.438800+0000","last_peered":"2024-03-29T04:07:41.438800+0000","last_clean":"2024-03-29T04:07:41.438800+0000","last_became_active":"2024-03-29T04:07:41.431503+0000","last_became_peered":"2024-03-29T04:07:41.431503+0000","last_unstale":"2024-03-29T04:07:41.438800+0000","last_undegraded":"2024-03-29T04:07:41.438800+0000","last_fullsized":"2024-03-29T04:07:41.438800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:23:32.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.447563+0000","last_change":"2024-03-29T04:07:40.418935+0000","last_active":"2024-03-29T04:07:41.447563+0000","last_peered":"2024-03-29T04:07:41.447563+0000","last_clean":"2024-03-29T04:07:41.447563+0000","last_became_active":"2024-03-29T04:07:40.418678+0000","last_became_peered":"2024-03-29T04:07:40.418678+0000","last_unstale":"2024-03-29T04:07:41.447563+0000","last_undegraded":"2024-03-29T04:07:41.447563+0000","last_fullsized":"2024-03-29T04:07:41.447563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:06:18.089762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.436056+0000","last_change":"2024-03-29T04:07:41.429488+0000","last_active":"2024-03-29T04:07:41.436056+0000","last_peered":"2024-03-29T04:07:41.436056+0000","last_clean":"2024-03-29T04:07:41.436056+0000","last_became_active":"2024-03-29T04:07:41.429233+0000","last_became_peered":"2024-03-29T04:07:41.429233+0000","last_unstale":"2024-03-29T04:07:41.436056+0000","last_undegraded":"2024-03-29T04:07:41.436056+0000","last_fullsized":"2024-03-29T04:07:41.436056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:12.561151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.426651+0000","last_change":"2024-03-29T04:07:40.429229+0000","last_active":"2024-03-29T04:07:41.426651+0000","last_peered":"2024-03-29T04:07:41.426651+0000","last_clean":"2024-03-29T04:07:41.426651+0000","last_became_active":"2024-03-29T04:07:40.428949+0000","last_became_peered":"2024-03-29T04:07:40.428949+0000","last_unstale":"2024-03-29T04:07:41.426651+0000","last_undegraded":"2024-03-29T04:07:41.426651+0000","last_fullsized":"2024-03-29T04:07:41.426651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:11.556289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.440609+0000","last_change":"2024-03-29T04:07:41.433525+0000","last_active":"2024-03-29T04:07:41.440609+0000","last_peered":"2024-03-29T04:07:41.440609+0000","last_clean":"2024-03-29T04:07:41.440609+0000","last_became_active":"2024-03-29T04:07:41.433128+0000","last_became_peered":"2024-03-29T04:07:41.433128+0000","last_unstale":"2024-03-29T04:07:41.440609+0000","last_undegraded":"2024-03-29T04:07:41.440609+0000","last_fullsized":"2024-03-29T04:07:41.440609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:40.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.416530+0000","last_change":"2024-03-29T04:07:40.419079+0000","last_active":"2024-03-29T04:07:42.416530+0000","last_peered":"2024-03-29T04:07:42.416530+0000","last_clean":"2024-03-29T04:07:42.416530+0000","last_became_active":"2024-03-29T04:07:40.418589+0000","last_became_peered":"2024-03-29T04:07:40.418589+0000","last_unstale":"2024-03-29T04:07:42.416530+0000","last_undegraded":"2024-03-29T04:07:42.416530+0000","last_fullsized":"2024-03-29T04:07:42.416530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:11.556289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417165+0000","last_change":"2024-03-29T04:07:41.425671+0000","last_active":"2024-03-29T04:07:42.417165+0000","last_peered":"2024-03-29T04:07:42.417165+0000","last_clean":"2024-03-29T04:07:42.417165+0000","last_became_active":"2024-03-29T04:07:41.425336+0000","last_became_peered":"2024-03-29T04:07:41.425336+0000","last_unstale":"2024-03-29T04:07:42.417165+0000","last_undegraded":"2024-03-29T04:07:42.417165+0000","last_fullsized":"2024-03-29T04:07:42.417165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:10:59.508554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.444043+0000","last_change":"2024-03-29T04:07:40.419997+0000","last_active":"2024-03-29T04:07:41.444043+0000","last_peered":"2024-03-29T04:07:41.444043+0000","last_clean":"2024-03-29T04:07:41.444043+0000","last_became_active":"2024-03-29T04:07:40.419725+0000","last_became_peered":"2024-03-29T04:07:40.419725+0000","last_unstale":"2024-03-29T04:07:41.444043+0000","last_undegraded":"2024-03-29T04:07:41.444043+0000","last_fullsized":"2024-03-29T04:07:41.444043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:40:12.105363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.435834+0000","last_change":"2024-03-29T04:07:41.429335+0000","last_active":"2024-03-29T04:07:41.435834+0000","last_peered":"2024-03-29T04:07:41.435834+0000","last_clean":"2024-03-29T04:07:41.435834+0000","last_became_active":"2024-03-29T04:07:41.428990+0000","last_became_peered":"2024-03-29T04:07:41.428990+0000","last_unstale":"2024-03-29T04:07:41.435834+0000","last_undegraded":"2024-03-29T04:07:41.435834+0000","last_fullsized":"2024-03-29T04:07:41.435834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:28:53.046219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.418057+0000","last_change":"2024-03-29T04:07:40.422355+0000","last_active":"2024-03-29T04:07:42.418057+0000","last_peered":"2024-03-29T04:07:42.418057+0000","last_clean":"2024-03-29T04:07:42.418057+0000","last_became_active":"2024-03-29T04:07:40.422081+0000","last_became_peered":"2024-03-29T04:07:40.422081+0000","last_unstale":"2024-03-29T04:07:42.418057+0000","last_undegraded":"2024-03-29T04:07:42.418057+0000","last_fullsized":"2024-03-29T04:07:42.418057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:42.944057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.435677+0000","last_change":"2024-03-29T04:07:41.428540+0000","last_active":"2024-03-29T04:07:41.435677+0000","last_peered":"2024-03-29T04:07:41.435677+0000","last_clean":"2024-03-29T04:07:41.435677+0000","last_became_active":"2024-03-29T04:07:41.428266+0000","last_became_peered":"2024-03-29T04:07:41.428266+0000","last_unstale":"2024-03-29T04:07:41.435677+0000","last_undegraded":"2024-03-29T04:07:41.435677+0000","last_fullsized":"2024-03-29T04:07:41.435677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:30:59.143986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.440462+0000","last_change":"2024-03-29T04:07:40.423940+0000","last_active":"2024-03-29T04:07:41.440462+0000","last_peered":"2024-03-29T04:07:41.440462+0000","last_clean":"2024-03-29T04:07:41.440462+0000","last_became_active":"2024-03-29T04:07:40.423675+0000","last_became_peered":"2024-03-29T04:07:40.423675+0000","last_unstale":"2024-03-29T04:07:41.440462+0000","last_undegraded":"2024-03-29T04:07:41.440462+0000","last_fullsized":"2024-03-29T04:07:41.440462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:08.909004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.436056+0000","last_change":"2024-03-29T04:07:41.429372+0000","last_active":"2024-03-29T04:07:41.436056+0000","last_peered":"2024-03-29T04:07:41.436056+0000","last_clean":"2024-03-29T04:07:41.436056+0000","last_became_active":"2024-03-29T04:07:41.429084+0000","last_became_peered":"2024-03-29T04:07:41.429084+0000","last_unstale":"2024-03-29T04:07:41.436056+0000","last_undegraded":"2024-03-29T04:07:41.436056+0000","last_fullsized":"2024-03-29T04:07:41.436056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:35.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413552+0000","last_change":"2024-03-29T04:07:40.420126+0000","last_active":"2024-03-29T04:07:42.413552+0000","last_peered":"2024-03-29T04:07:42.413552+0000","last_clean":"2024-03-29T04:07:42.413552+0000","last_became_active":"2024-03-29T04:07:40.419852+0000","last_became_peered":"2024-03-29T04:07:40.419852+0000","last_unstale":"2024-03-29T04:07:42.413552+0000","last_undegraded":"2024-03-29T04:07:42.413552+0000","last_fullsized":"2024-03-29T04:07:42.413552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:41.833725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.444483+0000","last_change":"2024-03-29T04:07:41.435921+0000","last_active":"2024-03-29T04:07:41.444483+0000","last_peered":"2024-03-29T04:07:41.444483+0000","last_clean":"2024-03-29T04:07:41.444483+0000","last_became_active":"2024-03-29T04:07:41.435681+0000","last_became_peered":"2024-03-29T04:07:41.435681+0000","last_unstale":"2024-03-29T04:07:41.444483+0000","last_undegraded":"2024-03-29T04:07:41.444483+0000","last_fullsized":"2024-03-29T04:07:41.444483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:36:03.632489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.417141+0000","last_change":"2024-03-29T04:07:40.430102+0000","last_active":"2024-03-29T04:07:41.417141+0000","last_peered":"2024-03-29T04:07:41.417141+0000","last_clean":"2024-03-29T04:07:41.417141+0000","last_became_active":"2024-03-29T04:07:40.429846+0000","last_became_peered":"2024-03-29T04:07:40.429846+0000","last_unstale":"2024-03-29T04:07:41.417141+0000","last_undegraded":"2024-03-29T04:07:41.417141+0000","last_fullsized":"2024-03-29T04:07:41.417141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.441903+0000","last_change":"2024-03-29T04:07:41.438935+0000","last_active":"2024-03-29T04:07:41.441903+0000","last_peered":"2024-03-29T04:07:41.441903+0000","last_clean":"2024-03-29T04:07:41.441903+0000","last_became_active":"2024-03-29T04:07:41.438629+0000","last_became_peered":"2024-03-29T04:07:41.438629+0000","last_unstale":"2024-03-29T04:07:41.441903+0000","last_undegraded":"2024-03-29T04:07:41.441903+0000","last_fullsized":"2024-03-29T04:07:41.441903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:08.125575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":168,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.415555+0000","last_change":"2024-03-29T04:07:29.856799+0000","last_active":"2024-03-29T04:07:41.415555+0000","last_peered":"2024-03-29T04:07:41.415555+0000","last_clean":"2024-03-29T04:07:41.415555+0000","last_became_active":"2024-03-29T04:07:29.855873+0000","last_became_peered":"2024-03-29T04:07:29.855873+0000","last_unstale":"2024-03-29T04:07:41.415555+0000","last_undegraded":"2024-03-29T04:07:41.415555+0000","last_fullsized":"2024-03-29T04:07:41.415555+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_clean_scrub_stamp":"2024-03-29T04:07:28.661516+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:25:15.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.422252+0000","last_change":"2024-03-29T04:07:40.427444+0000","last_active":"2024-03-29T04:07:41.422252+0000","last_peered":"2024-03-29T04:07:41.422252+0000","last_clean":"2024-03-29T04:07:41.422252+0000","last_became_active":"2024-03-29T04:07:40.426408+0000","last_became_peered":"2024-03-29T04:07:40.426408+0000","last_unstale":"2024-03-29T04:07:41.422252+0000","last_undegraded":"2024-03-29T04:07:41.422252+0000","last_fullsized":"2024-03-29T04:07:41.422252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:04.924835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.447655+0000","last_change":"2024-03-29T04:07:41.441256+0000","last_active":"2024-03-29T04:07:41.447655+0000","last_peered":"2024-03-29T04:07:41.447655+0000","last_clean":"2024-03-29T04:07:41.447655+0000","last_became_active":"2024-03-29T04:07:41.441002+0000","last_became_peered":"2024-03-29T04:07:41.441002+0000","last_unstale":"2024-03-29T04:07:41.447655+0000","last_undegraded":"2024-03-29T04:07:41.447655+0000","last_fullsized":"2024-03-29T04:07:41.447655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:10:59.508554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.417689+0000","last_change":"2024-03-29T04:07:40.427239+0000","last_active":"2024-03-29T04:07:41.417689+0000","last_peered":"2024-03-29T04:07:41.417689+0000","last_clean":"2024-03-29T04:07:41.417689+0000","last_became_active":"2024-03-29T04:07:40.426943+0000","last_became_peered":"2024-03-29T04:07:40.426943+0000","last_unstale":"2024-03-29T04:07:41.417689+0000","last_undegraded":"2024-03-29T04:07:41.417689+0000","last_fullsized":"2024-03-29T04:07:41.417689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:50:50.615524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.439698+0000","last_change":"2024-03-29T04:07:41.433014+0000","last_active":"2024-03-29T04:07:41.439698+0000","last_peered":"2024-03-29T04:07:41.439698+0000","last_clean":"2024-03-29T04:07:41.439698+0000","last_became_active":"2024-03-29T04:07:41.432678+0000","last_became_peered":"2024-03-29T04:07:41.432678+0000","last_unstale":"2024-03-29T04:07:41.439698+0000","last_undegraded":"2024-03-29T04:07:41.439698+0000","last_fullsized":"2024-03-29T04:07:41.439698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:42.838588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.416974+0000","last_change":"2024-03-29T04:07:40.421624+0000","last_active":"2024-03-29T04:07:41.416974+0000","last_peered":"2024-03-29T04:07:41.416974+0000","last_clean":"2024-03-29T04:07:41.416974+0000","last_became_active":"2024-03-29T04:07:40.421376+0000","last_became_peered":"2024-03-29T04:07:40.421376+0000","last_unstale":"2024-03-29T04:07:41.416974+0000","last_undegraded":"2024-03-29T04:07:41.416974+0000","last_fullsized":"2024-03-29T04:07:41.416974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:04.924835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413533+0000","last_change":"2024-03-29T04:07:41.421588+0000","last_active":"2024-03-29T04:07:42.413533+0000","last_peered":"2024-03-29T04:07:42.413533+0000","last_clean":"2024-03-29T04:07:42.413533+0000","last_became_active":"2024-03-29T04:07:41.421329+0000","last_became_peered":"2024-03-29T04:07:41.421329+0000","last_unstale":"2024-03-29T04:07:42.413533+0000","last_undegraded":"2024-03-29T04:07:42.413533+0000","last_fullsized":"2024-03-29T04:07:42.413533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:35.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.426788+0000","last_change":"2024-03-29T04:07:40.424596+0000","last_active":"2024-03-29T04:07:41.426788+0000","last_peered":"2024-03-29T04:07:41.426788+0000","last_clean":"2024-03-29T04:07:41.426788+0000","last_became_active":"2024-03-29T04:07:40.424288+0000","last_became_peered":"2024-03-29T04:07:40.424288+0000","last_unstale":"2024-03-29T04:07:41.426788+0000","last_undegraded":"2024-03-29T04:07:41.426788+0000","last_fullsized":"2024-03-29T04:07:41.426788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:34.960075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.418058+0000","last_change":"2024-03-29T04:07:41.429654+0000","last_active":"2024-03-29T04:07:42.418058+0000","last_peered":"2024-03-29T04:07:42.418058+0000","last_clean":"2024-03-29T04:07:42.418058+0000","last_became_active":"2024-03-29T04:07:41.429335+0000","last_became_peered":"2024-03-29T04:07:41.429335+0000","last_unstale":"2024-03-29T04:07:42.418058+0000","last_undegraded":"2024-03-29T04:07:42.418058+0000","last_fullsized":"2024-03-29T04:07:42.418058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:50:51.620387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413527+0000","last_change":"2024-03-29T04:07:40.420207+0000","last_active":"2024-03-29T04:07:42.413527+0000","last_peered":"2024-03-29T04:07:42.413527+0000","last_clean":"2024-03-29T04:07:42.413527+0000","last_became_active":"2024-03-29T04:07:40.419936+0000","last_became_peered":"2024-03-29T04:07:40.419936+0000","last_unstale":"2024-03-29T04:07:42.413527+0000","last_undegraded":"2024-03-29T04:07:42.413527+0000","last_fullsized":"2024-03-29T04:07:42.413527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:11.556289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.441036+0000","last_change":"2024-03-29T04:07:41.433871+0000","last_active":"2024-03-29T04:07:41.441036+0000","last_peered":"2024-03-29T04:07:41.441036+0000","last_clean":"2024-03-29T04:07:41.441036+0000","last_became_active":"2024-03-29T04:07:41.433183+0000","last_became_peered":"2024-03-29T04:07:41.433183+0000","last_unstale":"2024-03-29T04:07:41.441036+0000","last_undegraded":"2024-03-29T04:07:41.441036+0000","last_fullsized":"2024-03-29T04:07:41.441036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:42.838588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.418887+0000","last_change":"2024-03-29T04:07:40.432156+0000","last_active":"2024-03-29T04:07:41.418887+0000","last_peered":"2024-03-29T04:07:41.418887+0000","last_clean":"2024-03-29T04:07:41.418887+0000","last_became_active":"2024-03-29T04:07:40.431840+0000","last_became_peered":"2024-03-29T04:07:40.431840+0000","last_unstale":"2024-03-29T04:07:41.418887+0000","last_undegraded":"2024-03-29T04:07:41.418887+0000","last_fullsized":"2024-03-29T04:07:41.418887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:36:02.627626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413428+0000","last_change":"2024-03-29T04:07:41.421216+0000","last_active":"2024-03-29T04:07:42.413428+0000","last_peered":"2024-03-29T04:07:42.413428+0000","last_clean":"2024-03-29T04:07:42.413428+0000","last_became_active":"2024-03-29T04:07:41.420934+0000","last_became_peered":"2024-03-29T04:07:41.420934+0000","last_unstale":"2024-03-29T04:07:42.413428+0000","last_undegraded":"2024-03-29T04:07:42.413428+0000","last_fullsized":"2024-03-29T04:07:42.413428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:57.438333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.433094+0000","last_change":"2024-03-29T04:07:40.421842+0000","last_active":"2024-03-29T04:07:42.433094+0000","last_peered":"2024-03-29T04:07:42.433094+0000","last_clean":"2024-03-29T04:07:42.433094+0000","last_became_active":"2024-03-29T04:07:40.421590+0000","last_became_peered":"2024-03-29T04:07:40.421590+0000","last_unstale":"2024-03-29T04:07:42.433094+0000","last_undegraded":"2024-03-29T04:07:42.433094+0000","last_fullsized":"2024-03-29T04:07:42.433094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:08.909004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.432458+0000","last_change":"2024-03-29T04:07:41.429458+0000","last_active":"2024-03-29T04:07:41.432458+0000","last_peered":"2024-03-29T04:07:41.432458+0000","last_clean":"2024-03-29T04:07:41.432458+0000","last_became_active":"2024-03-29T04:07:41.429115+0000","last_became_peered":"2024-03-29T04:07:41.429115+0000","last_unstale":"2024-03-29T04:07:41.432458+0000","last_undegraded":"2024-03-29T04:07:41.432458+0000","last_fullsized":"2024-03-29T04:07:41.432458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:50:51.620387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417036+0000","last_change":"2024-03-29T04:07:40.422954+0000","last_active":"2024-03-29T04:07:42.417036+0000","last_peered":"2024-03-29T04:07:42.417036+0000","last_clean":"2024-03-29T04:07:42.417036+0000","last_became_active":"2024-03-29T04:07:40.422548+0000","last_became_peered":"2024-03-29T04:07:40.422548+0000","last_unstale":"2024-03-29T04:07:42.417036+0000","last_undegraded":"2024-03-29T04:07:42.417036+0000","last_fullsized":"2024-03-29T04:07:42.417036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:08.909004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.430336+0000","last_change":"2024-03-29T04:07:41.425089+0000","last_active":"2024-03-29T04:07:41.430336+0000","last_peered":"2024-03-29T04:07:41.430336+0000","last_clean":"2024-03-29T04:07:41.430336+0000","last_became_active":"2024-03-29T04:07:41.424814+0000","last_became_peered":"2024-03-29T04:07:41.424814+0000","last_unstale":"2024-03-29T04:07:41.430336+0000","last_undegraded":"2024-03-29T04:07:41.430336+0000","last_fullsized":"2024-03-29T04:07:41.430336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:40:13.110226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.415394+0000","last_change":"2024-03-29T04:07:40.429808+0000","last_active":"2024-03-29T04:07:41.415394+0000","last_peered":"2024-03-29T04:07:41.415394+0000","last_clean":"2024-03-29T04:07:41.415394+0000","last_became_active":"2024-03-29T04:07:40.428865+0000","last_became_peered":"2024-03-29T04:07:40.428865+0000","last_unstale":"2024-03-29T04:07:41.415394+0000","last_undegraded":"2024-03-29T04:07:41.415394+0000","last_fullsized":"2024-03-29T04:07:41.415394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:44.678749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417729+0000","last_change":"2024-03-29T04:07:41.434698+0000","last_active":"2024-03-29T04:07:42.417729+0000","last_peered":"2024-03-29T04:07:42.417729+0000","last_clean":"2024-03-29T04:07:42.417729+0000","last_became_active":"2024-03-29T04:07:41.434270+0000","last_became_peered":"2024-03-29T04:07:41.434270+0000","last_unstale":"2024-03-29T04:07:42.417729+0000","last_undegraded":"2024-03-29T04:07:42.417729+0000","last_fullsized":"2024-03-29T04:07:42.417729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:42.838588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413481+0000","last_change":"2024-03-29T04:07:40.425280+0000","last_active":"2024-03-29T04:07:42.413481+0000","last_peered":"2024-03-29T04:07:42.413481+0000","last_clean":"2024-03-29T04:07:42.413481+0000","last_became_active":"2024-03-29T04:07:40.424998+0000","last_became_peered":"2024-03-29T04:07:40.424998+0000","last_unstale":"2024-03-29T04:07:42.413481+0000","last_undegraded":"2024-03-29T04:07:42.413481+0000","last_fullsized":"2024-03-29T04:07:42.413481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:34.960075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.429884+0000","last_change":"2024-03-29T04:07:41.424267+0000","last_active":"2024-03-29T04:07:41.429884+0000","last_peered":"2024-03-29T04:07:41.429884+0000","last_clean":"2024-03-29T04:07:41.429884+0000","last_became_active":"2024-03-29T04:07:41.423920+0000","last_became_peered":"2024-03-29T04:07:41.423920+0000","last_unstale":"2024-03-29T04:07:41.429884+0000","last_undegraded":"2024-03-29T04:07:41.429884+0000","last_fullsized":"2024-03-29T04:07:41.429884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:09.913867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413416+0000","last_change":"2024-03-29T04:07:40.422602+0000","last_active":"2024-03-29T04:07:42.413416+0000","last_peered":"2024-03-29T04:07:42.413416+0000","last_clean":"2024-03-29T04:07:42.413416+0000","last_became_active":"2024-03-29T04:07:40.422332+0000","last_became_peered":"2024-03-29T04:07:40.422332+0000","last_unstale":"2024-03-29T04:07:42.413416+0000","last_undegraded":"2024-03-29T04:07:42.413416+0000","last_fullsized":"2024-03-29T04:07:42.413416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.432291+0000","last_change":"2024-03-29T04:07:41.427166+0000","last_active":"2024-03-29T04:07:41.432291+0000","last_peered":"2024-03-29T04:07:41.432291+0000","last_clean":"2024-03-29T04:07:41.432291+0000","last_became_active":"2024-03-29T04:07:41.426914+0000","last_became_peered":"2024-03-29T04:07:41.426914+0000","last_unstale":"2024-03-29T04:07:41.432291+0000","last_undegraded":"2024-03-29T04:07:41.432291+0000","last_fullsized":"2024-03-29T04:07:41.432291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:45.683611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.445027+0000","last_change":"2024-03-29T04:07:40.430896+0000","last_active":"2024-03-29T04:07:41.445027+0000","last_peered":"2024-03-29T04:07:41.445027+0000","last_clean":"2024-03-29T04:07:41.445027+0000","last_became_active":"2024-03-29T04:07:40.430549+0000","last_became_peered":"2024-03-29T04:07:40.430549+0000","last_unstale":"2024-03-29T04:07:41.445027+0000","last_undegraded":"2024-03-29T04:07:41.445027+0000","last_fullsized":"2024-03-29T04:07:41.445027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:39:46.902638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417189+0000","last_change":"2024-03-29T04:07:41.429537+0000","last_active":"2024-03-29T04:07:42.417189+0000","last_peered":"2024-03-29T04:07:42.417189+0000","last_clean":"2024-03-29T04:07:42.417189+0000","last_became_active":"2024-03-29T04:07:41.429173+0000","last_became_peered":"2024-03-29T04:07:41.429173+0000","last_unstale":"2024-03-29T04:07:42.417189+0000","last_undegraded":"2024-03-29T04:07:42.417189+0000","last_fullsized":"2024-03-29T04:07:42.417189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:57.438333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.444528+0000","last_change":"2024-03-29T04:07:41.436387+0000","last_active":"2024-03-29T04:07:41.444528+0000","last_peered":"2024-03-29T04:07:41.444528+0000","last_clean":"2024-03-29T04:07:41.444528+0000","last_became_active":"2024-03-29T04:07:41.436043+0000","last_became_peered":"2024-03-29T04:07:41.436043+0000","last_unstale":"2024-03-29T04:07:41.444528+0000","last_undegraded":"2024-03-29T04:07:41.444528+0000","last_fullsized":"2024-03-29T04:07:41.444528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:19.861306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.447698+0000","last_change":"2024-03-29T04:07:40.431266+0000","last_active":"2024-03-29T04:07:41.447698+0000","last_peered":"2024-03-29T04:07:41.447698+0000","last_clean":"2024-03-29T04:07:41.447698+0000","last_became_active":"2024-03-29T04:07:40.430984+0000","last_became_peered":"2024-03-29T04:07:40.430984+0000","last_unstale":"2024-03-29T04:07:41.447698+0000","last_undegraded":"2024-03-29T04:07:41.447698+0000","last_fullsized":"2024-03-29T04:07:41.447698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:41.833725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.444581+0000","last_change":"2024-03-29T04:07:41.436711+0000","last_active":"2024-03-29T04:07:41.444581+0000","last_peered":"2024-03-29T04:07:41.444581+0000","last_clean":"2024-03-29T04:07:41.444581+0000","last_became_active":"2024-03-29T04:07:41.436145+0000","last_became_peered":"2024-03-29T04:07:41.436145+0000","last_unstale":"2024-03-29T04:07:41.444581+0000","last_undegraded":"2024-03-29T04:07:41.444581+0000","last_fullsized":"2024-03-29T04:07:41.444581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:10:59.508554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.445084+0000","last_change":"2024-03-29T04:07:40.430774+0000","last_active":"2024-03-29T04:07:41.445084+0000","last_peered":"2024-03-29T04:07:41.445084+0000","last_clean":"2024-03-29T04:07:41.445084+0000","last_became_active":"2024-03-29T04:07:40.430503+0000","last_became_peered":"2024-03-29T04:07:40.430503+0000","last_unstale":"2024-03-29T04:07:41.445084+0000","last_undegraded":"2024-03-29T04:07:41.445084+0000","last_fullsized":"2024-03-29T04:07:41.445084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:37.084723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.443738+0000","last_change":"2024-03-29T04:07:41.441323+0000","last_active":"2024-03-29T04:07:41.443738+0000","last_peered":"2024-03-29T04:07:41.443738+0000","last_clean":"2024-03-29T04:07:41.443738+0000","last_became_active":"2024-03-29T04:07:41.441074+0000","last_became_peered":"2024-03-29T04:07:41.441074+0000","last_unstale":"2024-03-29T04:07:41.443738+0000","last_undegraded":"2024-03-29T04:07:41.443738+0000","last_fullsized":"2024-03-29T04:07:41.443738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:39:47.907501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.442350+0000","last_change":"2024-03-29T04:07:40.431979+0000","last_active":"2024-03-29T04:07:41.442350+0000","last_peered":"2024-03-29T04:07:41.442350+0000","last_clean":"2024-03-29T04:07:41.442350+0000","last_became_active":"2024-03-29T04:07:40.431702+0000","last_became_peered":"2024-03-29T04:07:40.431702+0000","last_unstale":"2024-03-29T04:07:41.442350+0000","last_undegraded":"2024-03-29T04:07:41.442350+0000","last_fullsized":"2024-03-29T04:07:41.442350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.432148+0000","last_change":"2024-03-29T04:07:41.427017+0000","last_active":"2024-03-29T04:07:41.432148+0000","last_peered":"2024-03-29T04:07:41.432148+0000","last_clean":"2024-03-29T04:07:41.432148+0000","last_became_active":"2024-03-29T04:07:41.426749+0000","last_became_peered":"2024-03-29T04:07:41.426749+0000","last_unstale":"2024-03-29T04:07:41.432148+0000","last_undegraded":"2024-03-29T04:07:41.432148+0000","last_fullsized":"2024-03-29T04:07:41.432148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:35.964938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-29T04:07:41.447710+0000","last_change":"2024-03-29T04:07:40.421536+0000","last_active":"2024-03-29T04:07:41.447710+0000","last_peered":"2024-03-29T04:07:41.447710+0000","last_clean":"2024-03-29T04:07:41.447710+0000","last_became_active":"2024-03-29T04:07:40.421279+0000","last_became_peered":"2024-03-29T04:07:40.421279+0000","last_unstale":"2024-03-29T04:07:41.447710+0000","last_undegraded":"2024-03-29T04:07:41.447710+0000","last_fullsized":"2024-03-29T04:07:41.447710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:43.704154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":64760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":200704,"data_stored":64794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:07:44.720 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T04:07:44.720 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-29T04:07:44.720 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-29T04:07:44.720 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-29T04:07:44.721 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-29T04:07:44.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.876+0000 7f343cd3d640 1 Processor -- start 2024-03-29T04:07:44.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.876+0000 7f343cd3d640 1 -- start start 2024-03-29T04:07:44.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.877+0000 7f343cd3d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f343815c930 0x7f343806fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.877+0000 7f343cd3d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f343806f430 0x7f34380cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.877+0000 7f343cd3d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f343806c520 0x7f343806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.877+0000 7f343cd3d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f3438072dc0 con 0x7f343815c930 2024-03-29T04:07:44.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.877+0000 7f343cd3d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f34380ce4b0 con 0x7f343806f430 2024-03-29T04:07:44.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.877+0000 7f343cd3d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f343806f8c0 con 0x7f343806c520 2024-03-29T04:07:44.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.877+0000 7f343659c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f343815c930 0x7f343806fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.877+0000 7f343659c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f343815c930 0x7f343806fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38702/0 (socket says 172.21.15.97:38702) 2024-03-29T04:07:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.877+0000 7f343659c640 1 -- 172.21.15.97:0/3730524615 learned_addr learned my addr 172.21.15.97:0/3730524615 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.877+0000 7f3436d9d640 1 --2- 172.21.15.97:0/3730524615 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f343806c520 0x7f343806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.877+0000 7f3435d9b640 1 --2- 172.21.15.97:0/3730524615 >> v2:172.21.15.154:3300/0 conn(0x7f343806f430 0x7f34380cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.878+0000 7f343659c640 1 -- 172.21.15.97:0/3730524615 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f343806c520 msgr2=0x7f343806c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.878+0000 7f343659c640 1 --2- 172.21.15.97:0/3730524615 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f343806c520 0x7f343806c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.878+0000 7f343659c640 1 -- 172.21.15.97:0/3730524615 >> v2:172.21.15.154:3300/0 conn(0x7f343806f430 msgr2=0x7f34380cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.878+0000 7f343659c640 1 --2- 172.21.15.97:0/3730524615 >> v2:172.21.15.154:3300/0 conn(0x7f343806f430 0x7f34380cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.878+0000 7f343659c640 1 -- 172.21.15.97:0/3730524615 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343806eba0 con 0x7f343815c930 2024-03-29T04:07:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.878+0000 7f3436d9d640 1 --2- 172.21.15.97:0/3730524615 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f343806c520 0x7f343806c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.878+0000 7f343659c640 1 --2- 172.21.15.97:0/3730524615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f343815c930 0x7f343806fb10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f34380b6d60 tx=0x7f342802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28c4a6de6a798ba server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.878+0000 7f343559a640 1 -- 172.21.15.97:0/3730524615 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342802c9e0 con 0x7f343815c930 2024-03-29T04:07:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.878+0000 7f343559a640 1 -- 172.21.15.97:0/3730524615 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f342802cb80 con 0x7f343815c930 2024-03-29T04:07:44.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.878+0000 7f343559a640 1 -- 172.21.15.97:0/3730524615 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34280343f0 con 0x7f343815c930 2024-03-29T04:07:44.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.879+0000 7f343cd3d640 1 -- 172.21.15.97:0/3730524615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f343815c930 msgr2=0x7f343806fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.879+0000 7f343cd3d640 1 --2- 172.21.15.97:0/3730524615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f343815c930 0x7f343806fb10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f34380b6d60 tx=0x7f342802be40 comp rx=0 tx=0).stop 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.879+0000 7f343cd3d640 1 -- 172.21.15.97:0/3730524615 shutdown_connections 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.879+0000 7f343cd3d640 1 --2- 172.21.15.97:0/3730524615 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f343806c520 0x7f343806c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.879+0000 7f343cd3d640 1 --2- 172.21.15.97:0/3730524615 >> v2:172.21.15.154:3300/0 conn(0x7f343806f430 0x7f34380cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.879+0000 7f343cd3d640 1 --2- 172.21.15.97:0/3730524615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f343815c930 0x7f343806fb10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.879+0000 7f343cd3d640 1 -- 172.21.15.97:0/3730524615 >> 172.21.15.97:0/3730524615 conn(0x7f3438072090 msgr2=0x7f34380724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.879+0000 7f343cd3d640 1 -- 172.21.15.97:0/3730524615 shutdown_connections 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.879+0000 7f343cd3d640 1 -- 172.21.15.97:0/3730524615 wait complete. 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.880+0000 7f343cd3d640 1 Processor -- start 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.880+0000 7f343cd3d640 1 -- start start 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f343cd3d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f343806c520 0x7f3438156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f343cd3d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f343815c930 0x7f343815ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f343cd3d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f343814d320 0x7f343814d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f343cd3d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34380b00c0 con 0x7f343814d320 2024-03-29T04:07:44.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f343cd3d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3438073010 con 0x7f343806c520 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f343cd3d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f34380b0240 con 0x7f343815c930 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f3436d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f343814d320 0x7f343814d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f3436d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f343814d320 0x7f343814d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38718/0 (socket says 172.21.15.97:38718) 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f3436d9d640 1 -- 172.21.15.97:0/3401382463 learned_addr learned my addr 172.21.15.97:0/3401382463 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f343659c640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f343806c520 0x7f3438156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f3435d9b640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f343815c930 0x7f343815ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f3436d9d640 1 -- 172.21.15.97:0/3401382463 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f343815c930 msgr2=0x7f343815ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f3436d9d640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f343815c930 0x7f343815ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f3436d9d640 1 -- 172.21.15.97:0/3401382463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f343806c520 msgr2=0x7f3438156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f3436d9d640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f343806c520 0x7f3438156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.881+0000 7f3436d9d640 1 -- 172.21.15.97:0/3401382463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3438158ab0 con 0x7f343814d320 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.882+0000 7f3436d9d640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f343814d320 0x7f343814d790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f343000e860 tx=0x7f343000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.882+0000 7f3426ffd640 1 -- 172.21.15.97:0/3401382463 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430017070 con 0x7f343814d320 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.882+0000 7f3426ffd640 1 -- 172.21.15.97:0/3401382463 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3430004020 con 0x7f343814d320 2024-03-29T04:07:44.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.882+0000 7f343cd3d640 1 -- 172.21.15.97:0/3401382463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3438158d90 con 0x7f343814d320 2024-03-29T04:07:44.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.882+0000 7f3426ffd640 1 -- 172.21.15.97:0/3401382463 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34300043f0 con 0x7f343814d320 2024-03-29T04:07:44.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.882+0000 7f343cd3d640 1 -- 172.21.15.97:0/3401382463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3438151960 con 0x7f343814d320 2024-03-29T04:07:44.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.883+0000 7f343cd3d640 1 -- 172.21.15.97:0/3401382463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3438068640 con 0x7f343814d320 2024-03-29T04:07:44.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.883+0000 7f3426ffd640 1 -- 172.21.15.97:0/3401382463 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f343000c8e0 con 0x7f343814d320 2024-03-29T04:07:44.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.884+0000 7f3426ffd640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f341806c530 0x7f341806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:44.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.884+0000 7f3426ffd640 1 -- 172.21.15.97:0/3401382463 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f343008b750 con 0x7f343814d320 2024-03-29T04:07:44.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.884+0000 7f343659c640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f341806c530 0x7f341806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:44.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.884+0000 7f343659c640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f341806c530 0x7f341806e9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3428035000 tx=0x7f3428030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:44.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:44.888+0000 7f3426ffd640 1 -- 172.21.15.97:0/3401382463 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f34300597b0 con 0x7f343814d320 2024-03-29T04:07:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.110+0000 7f343cd3d640 1 -- 172.21.15.97:0/3401382463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f343806fad0 con 0x7f343814d320 2024-03-29T04:07:45.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.113+0000 7f3426ffd640 1 -- 172.21.15.97:0/3401382463 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+173733 (secure 0 0 0) 0x7f343005dab0 con 0x7f343814d320 2024-03-29T04:07:45.095 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:45.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.115+0000 7f343cd3d640 1 -- 172.21.15.97:0/3401382463 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f341806c530 msgr2=0x7f341806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.115+0000 7f343cd3d640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f341806c530 0x7f341806e9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3428035000 tx=0x7f3428030000 comp rx=0 tx=0).stop 2024-03-29T04:07:45.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.115+0000 7f343cd3d640 1 -- 172.21.15.97:0/3401382463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f343814d320 msgr2=0x7f343814d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.115+0000 7f343cd3d640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f343814d320 0x7f343814d790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f343000e860 tx=0x7f343000ed20 comp rx=0 tx=0).stop 2024-03-29T04:07:45.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.115+0000 7f343cd3d640 1 -- 172.21.15.97:0/3401382463 shutdown_connections 2024-03-29T04:07:45.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.115+0000 7f343cd3d640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f341806c530 0x7f341806e9f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.115+0000 7f343cd3d640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f343814d320 0x7f343814d790 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.115+0000 7f343cd3d640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f343815c930 0x7f343815ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.115+0000 7f343cd3d640 1 --2- 172.21.15.97:0/3401382463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f343806c520 0x7f3438156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.115+0000 7f343cd3d640 1 -- 172.21.15.97:0/3401382463 >> 172.21.15.97:0/3401382463 conn(0x7f3438072090 msgr2=0x7f34380cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.115+0000 7f343cd3d640 1 -- 172.21.15.97:0/3401382463 shutdown_connections 2024-03-29T04:07:45.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.115+0000 7f343cd3d640 1 -- 172.21.15.97:0/3401382463 wait complete. 2024-03-29T04:07:45.106 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2286367679},{"type":"v1","addr":"172.21.15.97:6833","nonce":2286367679}]},"active_addr":"172.21.15.97:6833/2286367679","active_change":"2024-03-29T04:07:20.629333+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4112,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3786467214}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":765141273}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3406385453}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":125866691}]}]} 2024-03-29T04:07:45.110 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-29T04:07:45.110 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-29T04:07:45.110 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:45.174 INFO:tasks.ceph.mds.d.smithi154.stderr:2024-03-29T04:07:45.195+0000 7f8da22aa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T04:07:45.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.230+0000 7ff0dbe88640 1 Processor -- start 2024-03-29T04:07:45.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.231+0000 7ff0dbe88640 1 -- start start 2024-03-29T04:07:45.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.231+0000 7ff0dbe88640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff0d406d030 0x7ff0d409b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.231+0000 7ff0dbe88640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0d40d0f20 con 0x7ff0d406d410 2024-03-29T04:07:45.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.231+0000 7ff0dbe88640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0d40d1200 con 0x7ff0d40ccd00 2024-03-29T04:07:45.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.231+0000 7ff0dbe88640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff0d4162250 con 0x7ff0d406d030 2024-03-29T04:07:45.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.232+0000 7ff0d9c24640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff0d406d030 0x7ff0d409b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.232+0000 7ff0d9423640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff0d406d410 0x7ff0d40d0610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50356/0 (socket says 172.21.15.97:50356) 2024-03-29T04:07:45.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.232+0000 7ff0d9423640 1 -- 172.21.15.97:0/92563644 learned_addr learned my addr 172.21.15.97:0/92563644 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:45.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.232+0000 7ff0d9c24640 1 -- 172.21.15.97:0/92563644 >> v1:172.21.15.154:6789/0 conn(0x7ff0d40ccd00 legacy=0x7ff0d40cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.232+0000 7ff0d9c24640 1 -- 172.21.15.97:0/92563644 >> v1:172.21.15.97:6789/0 conn(0x7ff0d406d410 legacy=0x7ff0d40d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.232+0000 7ff0d8c22640 1 -- 172.21.15.97:0/92563644 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1181291670 0 0) 0x7ff0d40d1200 con 0x7ff0d40ccd00 2024-03-29T04:07:45.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.232+0000 7ff0d9c24640 1 -- 172.21.15.97:0/92563644 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d40d14e0 con 0x7ff0d406d030 2024-03-29T04:07:45.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.233+0000 7ff0d9c24640 1 --2- 172.21.15.97:0/92563644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d406d030 0x7ff0d409b870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40097a0 tx=0x7ff0c402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28d07d0c7c6f9c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.233+0000 7ff0d8c22640 1 -- 172.21.15.97:0/92563644 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4037070 con 0x7ff0d406d030 2024-03-29T04:07:45.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.233+0000 7ff0d8c22640 1 -- 172.21.15.97:0/92563644 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c4030710 con 0x7ff0d406d030 2024-03-29T04:07:45.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.233+0000 7ff0d8c22640 1 -- 172.21.15.97:0/92563644 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4030ae0 con 0x7ff0d406d030 2024-03-29T04:07:45.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.233+0000 7ff0dbe88640 1 -- 172.21.15.97:0/92563644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d406d030 msgr2=0x7ff0d409b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.233+0000 7ff0dbe88640 1 --2- 172.21.15.97:0/92563644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d406d030 0x7ff0d409b870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40097a0 tx=0x7ff0c402b790 comp rx=0 tx=0).stop 2024-03-29T04:07:45.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.233+0000 7ff0dbe88640 1 -- 172.21.15.97:0/92563644 shutdown_connections 2024-03-29T04:07:45.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.233+0000 7ff0dbe88640 1 --2- 172.21.15.97:0/92563644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d406d030 0x7ff0d409b870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.233+0000 7ff0dbe88640 1 -- 172.21.15.97:0/92563644 >> 172.21.15.97:0/92563644 conn(0x7ff0d40b3f00 msgr2=0x7ff0d40b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.233+0000 7ff0dbe88640 1 -- 172.21.15.97:0/92563644 shutdown_connections 2024-03-29T04:07:45.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.233+0000 7ff0dbe88640 1 -- 172.21.15.97:0/92563644 wait complete. 2024-03-29T04:07:45.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.234+0000 7ff0dbe88640 1 Processor -- start 2024-03-29T04:07:45.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.234+0000 7ff0dbe88640 1 -- start start 2024-03-29T04:07:45.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0dbe88640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d406d030 0x7ff0d40cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0dbe88640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d406d410 0x7ff0d40cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0dbe88640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0d40ccd00 0x7ff0d415f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0dbe88640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d406d8a0 con 0x7ff0d406d410 2024-03-29T04:07:45.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0dbe88640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d40d1aa0 con 0x7ff0d406d030 2024-03-29T04:07:45.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0dbe88640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d406da20 con 0x7ff0d40ccd00 2024-03-29T04:07:45.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0d9423640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d406d410 0x7ff0d40cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0d9423640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d406d410 0x7ff0d40cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38742/0 (socket says 172.21.15.97:38742) 2024-03-29T04:07:45.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0d9423640 1 -- 172.21.15.97:0/3252437913 learned_addr learned my addr 172.21.15.97:0/3252437913 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0d9c24640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d406d030 0x7ff0d40cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0da425640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0d40ccd00 0x7ff0d415f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0d9423640 1 -- 172.21.15.97:0/3252437913 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0d40ccd00 msgr2=0x7ff0d415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0d9423640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0d40ccd00 0x7ff0d415f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0d9423640 1 -- 172.21.15.97:0/3252437913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d406d030 msgr2=0x7ff0d40cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0d9423640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d406d030 0x7ff0d40cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.235+0000 7ff0d9423640 1 -- 172.21.15.97:0/3252437913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d415f960 con 0x7ff0d406d410 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.236+0000 7ff0d9423640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d406d410 0x7ff0d40cc530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000c780 tx=0x7ff0d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.236+0000 7ff0ca7fc640 1 -- 172.21.15.97:0/3252437913 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0018070 con 0x7ff0d406d410 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.236+0000 7ff0ca7fc640 1 -- 172.21.15.97:0/3252437913 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d00072d0 con 0x7ff0d406d410 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.236+0000 7ff0dbe88640 1 -- 172.21.15.97:0/3252437913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d40cbcf0 con 0x7ff0d406d410 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.236+0000 7ff0ca7fc640 1 -- 172.21.15.97:0/3252437913 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00047e0 con 0x7ff0d406d410 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.236+0000 7ff0dbe88640 1 -- 172.21.15.97:0/3252437913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d415e940 con 0x7ff0d406d410 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.237+0000 7ff0dbe88640 1 -- 172.21.15.97:0/3252437913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d4068d50 con 0x7ff0d406d410 2024-03-29T04:07:45.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.237+0000 7ff0ca7fc640 1 -- 172.21.15.97:0/3252437913 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff0d001b070 con 0x7ff0d406d410 2024-03-29T04:07:45.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.238+0000 7ff0ca7fc640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff0b806c530 0x7ff0b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.238+0000 7ff0d9c24640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff0b806c530 0x7ff0b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.238+0000 7ff0ca7fc640 1 -- 172.21.15.97:0/3252437913 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff0d008aa50 con 0x7ff0d406d410 2024-03-29T04:07:45.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.238+0000 7ff0d9c24640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff0b806c530 0x7ff0b806e9f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4034000 tx=0x7ff0c402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.242+0000 7ff0ca7fc640 1 -- 172.21.15.97:0/3252437913 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff0d0058a20 con 0x7ff0d406d410 2024-03-29T04:07:45.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.417+0000 7ff0dbe88640 1 -- 172.21.15.97:0/3252437913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0d40c8e70 con 0x7ff0d406d410 2024-03-29T04:07:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.418+0000 7ff0ca7fc640 1 -- 172.21.15.97:0/3252437913 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7ff0d005cd20 con 0x7ff0d406d410 2024-03-29T04:07:45.398 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:45.398 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.420+0000 7ff0dbe88640 1 -- 172.21.15.97:0/3252437913 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff0b806c530 msgr2=0x7ff0b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.420+0000 7ff0dbe88640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff0b806c530 0x7ff0b806e9f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4034000 tx=0x7ff0c402f000 comp rx=0 tx=0).stop 2024-03-29T04:07:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.420+0000 7ff0dbe88640 1 -- 172.21.15.97:0/3252437913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d406d410 msgr2=0x7ff0d40cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.420+0000 7ff0dbe88640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d406d410 0x7ff0d40cc530 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000c780 tx=0x7ff0d000cc40 comp rx=0 tx=0).stop 2024-03-29T04:07:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.420+0000 7ff0dbe88640 1 -- 172.21.15.97:0/3252437913 shutdown_connections 2024-03-29T04:07:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.420+0000 7ff0dbe88640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff0b806c530 0x7ff0b806e9f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.420+0000 7ff0dbe88640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0d40ccd00 0x7ff0d415f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.420+0000 7ff0dbe88640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0d406d410 0x7ff0d40cc530 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.420+0000 7ff0dbe88640 1 --2- 172.21.15.97:0/3252437913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d406d030 0x7ff0d40cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.420+0000 7ff0dbe88640 1 -- 172.21.15.97:0/3252437913 >> 172.21.15.97:0/3252437913 conn(0x7ff0d40b3f00 msgr2=0x7ff0d406faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.420+0000 7ff0dbe88640 1 -- 172.21.15.97:0/3252437913 shutdown_connections 2024-03-29T04:07:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.420+0000 7ff0dbe88640 1 -- 172.21.15.97:0/3252437913 wait complete. 2024-03-29T04:07:45.411 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-29T04:07:45.411 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.530+0000 7fa8badf0640 1 Processor -- start 2024-03-29T04:07:45.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.531+0000 7fa8badf0640 1 -- start start 2024-03-29T04:07:45.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.531+0000 7fa8badf0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa8b415c790 0x7fa8b40d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.531+0000 7fa8badf0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa8b40ce120 0x7fa8b40d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.531+0000 7fa8badf0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8b415cb70 con 0x7fa8b409aa30 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.531+0000 7fa8badf0640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa8b409ba70 con 0x7fa8b40ce120 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.531+0000 7fa8badf0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fa8b40cdc90 con 0x7fa8b415c790 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.532+0000 7fa8b3fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa8b40ce120 0x7fa8b40d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.532+0000 7fa8b3fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa8b40ce120 0x7fa8b40d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38744/0 (socket says 172.21.15.97:38744) 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.532+0000 7fa8b3fff640 1 -- 172.21.15.97:0/101387241 learned_addr learned my addr 172.21.15.97:0/101387241 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.532+0000 7fa8b8b8c640 1 --2- 172.21.15.97:0/101387241 >> v2:172.21.15.154:3300/0 conn(0x7fa8b415c790 0x7fa8b40d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.532+0000 7fa8b3fff640 1 -- 172.21.15.97:0/101387241 >> v2:172.21.15.154:3300/0 conn(0x7fa8b415c790 msgr2=0x7fa8b40d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.532+0000 7fa8b3fff640 1 --2- 172.21.15.97:0/101387241 >> v2:172.21.15.154:3300/0 conn(0x7fa8b415c790 0x7fa8b40d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.532+0000 7fa8b3fff640 1 -- 172.21.15.97:0/101387241 >> v1:172.21.15.154:6789/0 conn(0x7fa8b409aa30 legacy=0x7fa8b4160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.532+0000 7fa8b3fff640 1 -- 172.21.15.97:0/101387241 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b40cd080 con 0x7fa8b40ce120 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.533+0000 7fa8b3fff640 1 --2- 172.21.15.97:0/101387241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8b40ce120 0x7fa8b40d0f30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa8b409b6f0 tx=0x7fa8a802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae514ac1fcdb39c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.533+0000 7fa8b37fe640 1 -- 172.21.15.97:0/101387241 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8034070 con 0x7fa8b40ce120 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.533+0000 7fa8b37fe640 1 -- 172.21.15.97:0/101387241 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a8030860 con 0x7fa8b40ce120 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.533+0000 7fa8b37fe640 1 -- 172.21.15.97:0/101387241 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8030c30 con 0x7fa8b40ce120 2024-03-29T04:07:45.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.533+0000 7fa8badf0640 1 -- 172.21.15.97:0/101387241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8b40ce120 msgr2=0x7fa8b40d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.533+0000 7fa8badf0640 1 --2- 172.21.15.97:0/101387241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8b40ce120 0x7fa8b40d0f30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa8b409b6f0 tx=0x7fa8a802b9a0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.533+0000 7fa8badf0640 1 -- 172.21.15.97:0/101387241 shutdown_connections 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.533+0000 7fa8badf0640 1 --2- 172.21.15.97:0/101387241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8b40ce120 0x7fa8b40d0f30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.533+0000 7fa8badf0640 1 --2- 172.21.15.97:0/101387241 >> v2:172.21.15.154:3300/0 conn(0x7fa8b415c790 0x7fa8b40d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.533+0000 7fa8badf0640 1 -- 172.21.15.97:0/101387241 >> 172.21.15.97:0/101387241 conn(0x7fa8b40b3f00 msgr2=0x7fa8b40b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.533+0000 7fa8badf0640 1 -- 172.21.15.97:0/101387241 shutdown_connections 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.533+0000 7fa8badf0640 1 -- 172.21.15.97:0/101387241 wait complete. 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.534+0000 7fa8badf0640 1 Processor -- start 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.534+0000 7fa8badf0640 1 -- start start 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.535+0000 7fa8badf0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8b409aa30 0x7fa8b40b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.535+0000 7fa8badf0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8b40ce120 0x7fa8b41f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.535+0000 7fa8badf0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8b415c790 0x7fa8b4148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.535+0000 7fa8badf0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b40cd920 con 0x7fa8b415c790 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.535+0000 7fa8badf0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b40cd640 con 0x7fa8b40ce120 2024-03-29T04:07:45.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.535+0000 7fa8badf0640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b409aea0 con 0x7fa8b409aa30 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.535+0000 7fa8b938d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8b415c790 0x7fa8b4148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.535+0000 7fa8b938d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8b415c790 0x7fa8b4148120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38754/0 (socket says 172.21.15.97:38754) 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.535+0000 7fa8b938d640 1 -- 172.21.15.97:0/1358445828 learned_addr learned my addr 172.21.15.97:0/1358445828 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.535+0000 7fa8b8b8c640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8b409aa30 0x7fa8b40b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.535+0000 7fa8b3fff640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8b40ce120 0x7fa8b41f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.536+0000 7fa8b938d640 1 -- 172.21.15.97:0/1358445828 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8b409aa30 msgr2=0x7fa8b40b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.536+0000 7fa8b938d640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8b409aa30 0x7fa8b40b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.536+0000 7fa8b938d640 1 -- 172.21.15.97:0/1358445828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8b40ce120 msgr2=0x7fa8b41f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.536+0000 7fa8b938d640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8b40ce120 0x7fa8b41f5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.536+0000 7fa8b938d640 1 -- 172.21.15.97:0/1358445828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b40c6d60 con 0x7fa8b415c790 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.536+0000 7fa8b938d640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8b415c790 0x7fa8b4148120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400c3f0 tx=0x7fa8a400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.536+0000 7fa8b17fa640 1 -- 172.21.15.97:0/1358445828 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a40052d0 con 0x7fa8b415c790 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.536+0000 7fa8b17fa640 1 -- 172.21.15.97:0/1358445828 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a4005490 con 0x7fa8b415c790 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.536+0000 7fa8badf0640 1 -- 172.21.15.97:0/1358445828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b40b57a0 con 0x7fa8b415c790 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.536+0000 7fa8b17fa640 1 -- 172.21.15.97:0/1358445828 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4007260 con 0x7fa8b415c790 2024-03-29T04:07:45.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.536+0000 7fa8badf0640 1 -- 172.21.15.97:0/1358445828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b40c6fe0 con 0x7fa8b415c790 2024-03-29T04:07:45.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.537+0000 7fa8badf0640 1 -- 172.21.15.97:0/1358445828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b4152090 con 0x7fa8b415c790 2024-03-29T04:07:45.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.537+0000 7fa8b17fa640 1 -- 172.21.15.97:0/1358445828 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa8a4007400 con 0x7fa8b415c790 2024-03-29T04:07:45.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.538+0000 7fa8b17fa640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa8a006c530 0x7fa8a006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.538+0000 7fa8b17fa640 1 -- 172.21.15.97:0/1358445828 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa8a408aa20 con 0x7fa8b415c790 2024-03-29T04:07:45.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.538+0000 7fa8b8b8c640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa8a006c530 0x7fa8a006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.538+0000 7fa8b8b8c640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa8a006c530 0x7fa8a006e9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40b8490 tx=0x7fa8ac00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.542+0000 7fa8b17fa640 1 -- 172.21.15.97:0/1358445828 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa8b40c6d60 con 0x7fa8b415c790 2024-03-29T04:07:45.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.719+0000 7fa8badf0640 1 -- 172.21.15.97:0/1358445828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8b40b4ab0 con 0x7fa8b415c790 2024-03-29T04:07:45.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.720+0000 7fa8b17fa640 1 -- 172.21.15.97:0/1358445828 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fa8a4058a80 con 0x7fa8b415c790 2024-03-29T04:07:45.700 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:45.701 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:45.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.722+0000 7fa8badf0640 1 -- 172.21.15.97:0/1358445828 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa8a006c530 msgr2=0x7fa8a006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.722+0000 7fa8badf0640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa8a006c530 0x7fa8a006e9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40b8490 tx=0x7fa8ac00b040 comp rx=0 tx=0).stop 2024-03-29T04:07:45.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.722+0000 7fa8badf0640 1 -- 172.21.15.97:0/1358445828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8b415c790 msgr2=0x7fa8b4148120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.722+0000 7fa8badf0640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8b415c790 0x7fa8b4148120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400c3f0 tx=0x7fa8a400c8b0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.722+0000 7fa8badf0640 1 -- 172.21.15.97:0/1358445828 shutdown_connections 2024-03-29T04:07:45.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.722+0000 7fa8badf0640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa8a006c530 0x7fa8a006e9f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.722+0000 7fa8badf0640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8b415c790 0x7fa8b4148120 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.722+0000 7fa8badf0640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8b40ce120 0x7fa8b41f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.722+0000 7fa8badf0640 1 --2- 172.21.15.97:0/1358445828 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa8b409aa30 0x7fa8b40b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.722+0000 7fa8badf0640 1 -- 172.21.15.97:0/1358445828 >> 172.21.15.97:0/1358445828 conn(0x7fa8b40b3f00 msgr2=0x7fa8b40af140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.723+0000 7fa8badf0640 1 -- 172.21.15.97:0/1358445828 shutdown_connections 2024-03-29T04:07:45.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.723+0000 7fa8badf0640 1 -- 172.21.15.97:0/1358445828 wait complete. 2024-03-29T04:07:45.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-29T04:07:45.714 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-29T04:07:45.714 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-29T04:07:45.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-29T04:07:45.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-29T04:07:45.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-29T04:07:45.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-29T04:07:45.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-29T04:07:45.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.902+0000 7fcb5b51b640 1 Processor -- start 2024-03-29T04:07:45.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.902+0000 7fcb5b51b640 1 -- start start 2024-03-29T04:07:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.903+0000 7fcb5b51b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcb540cfde0 0x7fcb540d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.903+0000 7fcb5b51b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb5415ed00 con 0x7fcb540d0f90 2024-03-29T04:07:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.903+0000 7fcb5b51b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb540d21e0 con 0x7fcb5415e920 2024-03-29T04:07:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.903+0000 7fcb5b51b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fcb540d2480 con 0x7fcb540cfde0 2024-03-29T04:07:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.903+0000 7fcb592b7640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fcb5415e920 0x7fcb540d3240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:40404/0 (socket says 172.21.15.97:40404) 2024-03-29T04:07:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.903+0000 7fcb592b7640 1 -- 172.21.15.97:0/2588919030 learned_addr learned my addr 172.21.15.97:0/2588919030 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb58ab6640 1 --2- 172.21.15.97:0/2588919030 >> v2:172.21.15.154:3300/0 conn(0x7fcb540cfde0 0x7fcb540d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604052716 0 0) 0x7fcb540d21e0 con 0x7fcb5415e920 2024-03-29T04:07:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb40003520 con 0x7fcb5415e920 2024-03-29T04:07:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2821223481 0 0) 0x7fcb5415ed00 con 0x7fcb540d0f90 2024-03-29T04:07:45.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb540d21e0 con 0x7fcb540d0f90 2024-03-29T04:07:45.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7f49bc115640 1 Processor -- start 2024-03-29T04:07:45.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7efc70b15640 1 Processor -- start 2024-03-29T04:07:45.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7efc70b15640 1 -- start start 2024-03-29T04:07:45.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1073316815 0 0) 0x7fcb540d21e0 con 0x7fcb540d0f90 2024-03-29T04:07:45.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 >> v2:172.21.15.154:3300/0 conn(0x7fcb540cfde0 msgr2=0x7fcb540d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:45.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 --2- 172.21.15.97:0/2588919030 >> v2:172.21.15.154:3300/0 conn(0x7fcb540cfde0 0x7fcb540d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 >> v1:172.21.15.154:6789/0 conn(0x7fcb5415e920 legacy=0x7fcb540d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb540d2760 con 0x7fcb540d0f90 2024-03-29T04:07:45.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7fcb4c0019f0 con 0x7fcb540d0f90 2024-03-29T04:07:45.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 >> v1:172.21.15.97:6789/0 conn(0x7fcb540d0f90 legacy=0x7fcb5406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 --2- 172.21.15.97:0/2588919030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb40004ba0 0x7fcb40004fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 --2- 172.21.15.97:0/2588919030 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb400037a0 0x7fcb40003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.904+0000 7fcb4bfff640 1 --2- 172.21.15.97:0/2588919030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb40004160 0x7fcb4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcb540d2480 con 0x7fcb40004ba0 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4000c040 con 0x7fcb40004160 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4000c240 con 0x7fcb400037a0 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb592b7640 1 --2- 172.21.15.97:0/2588919030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb40004ba0 0x7fcb40004fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb59ab8640 1 --2- 172.21.15.97:0/2588919030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb40004160 0x7fcb4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb58ab6640 1 --2- 172.21.15.97:0/2588919030 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb400037a0 0x7fcb40003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb592b7640 1 -- 172.21.15.97:0/2588919030 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb400037a0 msgr2=0x7fcb40003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb592b7640 1 --2- 172.21.15.97:0/2588919030 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb400037a0 0x7fcb40003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb592b7640 1 -- 172.21.15.97:0/2588919030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb40004160 msgr2=0x7fcb4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb592b7640 1 --2- 172.21.15.97:0/2588919030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb40004160 0x7fcb4000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb592b7640 1 -- 172.21.15.97:0/2588919030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb540d2760 con 0x7fcb40004ba0 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb59ab8640 1 --2- 172.21.15.97:0/2588919030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb40004160 0x7fcb4000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb59ab8640 1 -- 172.21.15.97:0/2588919030 reap_dead start 2024-03-29T04:07:45.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7fcb58ab6640 1 --2- 172.21.15.97:0/2588919030 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb400037a0 0x7fcb40003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:07:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.905+0000 7f49bc115640 1 -- start start 2024-03-29T04:07:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.906+0000 7f49bc115640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f49b40b3cc0 0x7f49b40d1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.906+0000 7f49bc115640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49b406c720 0x7f49b40d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.906+0000 7f49bc115640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b40cf170 con 0x7f49b40d2c80 2024-03-29T04:07:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.906+0000 7f49bc115640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f49b40d5480 con 0x7f49b406c720 2024-03-29T04:07:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.906+0000 7f49bc115640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f49b40c77f0 con 0x7f49b40b3cc0 2024-03-29T04:07:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.906+0000 7f49ba6b2640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f49b40d2c80 0x7f49b40d30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50380/0 (socket says 172.21.15.97:50380) 2024-03-29T04:07:45.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.906+0000 7f49ba6b2640 1 -- 172.21.15.97:0/1432966571 learned_addr learned my addr 172.21.15.97:0/1432966571 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:45.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7fcb592b7640 1 --2- 172.21.15.97:0/2588919030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb40004ba0 0x7fcb40004fa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcb500019c0 tx=0x7fcb40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c423fc391c7b6d28 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7f79be70e640 1 Processor -- start 2024-03-29T04:07:45.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7efc70b15640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc64009180 0x7efc64052ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7f79be70e640 1 -- start start 2024-03-29T04:07:45.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7f49b9eb1640 1 --2- 172.21.15.97:0/1432966571 >> v2:172.21.15.154:3300/0 conn(0x7f49b40b3cc0 0x7f49b40d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7f49b96b0640 1 --2- 172.21.15.97:0/1432966571 >> v2:172.21.15.97:3300/0 conn(0x7f49b406c720 0x7f49b40d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7fdc95bcd640 1 Processor -- start 2024-03-29T04:07:45.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7fdc95bcd640 1 -- start start 2024-03-29T04:07:45.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50036070 con 0x7fcb40004ba0 2024-03-29T04:07:45.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb50004aa0 con 0x7fcb40004ba0 2024-03-29T04:07:45.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7fcb4bfff640 1 -- 172.21.15.97:0/2588919030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50001610 con 0x7fcb40004ba0 2024-03-29T04:07:45.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7f49b96b0640 1 -- 172.21.15.97:0/1432966571 >> v2:172.21.15.154:3300/0 conn(0x7f49b40b3cc0 msgr2=0x7f49b40d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:45.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7fc753fff640 1 Processor -- start 2024-03-29T04:07:45.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7f49b96b0640 1 --2- 172.21.15.97:0/1432966571 >> v2:172.21.15.154:3300/0 conn(0x7f49b40b3cc0 0x7f49b40d1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7f49b96b0640 1 -- 172.21.15.97:0/1432966571 >> v1:172.21.15.97:6789/0 conn(0x7f49b40d2c80 legacy=0x7f49b40d30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7f49b96b0640 1 -- 172.21.15.97:0/1432966571 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b406cb80 con 0x7f49b406c720 2024-03-29T04:07:45.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7f79be70e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f79b815d140 0x7f79b806f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7f79be70e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b8163640 con 0x7f79b8163180 2024-03-29T04:07:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7efc70b15640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efc64048c30 0x7efc64049050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7efc70b15640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc640eb290 0x7efc640eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7efc70b15640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7efc64049590 con 0x7efc64048c30 2024-03-29T04:07:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7efc70b15640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7efc64009560 con 0x7efc640eb290 2024-03-29T04:07:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7efc70b15640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7efc64035960 con 0x7efc64009180 2024-03-29T04:07:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7efc6ad9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc640eb290 0x7efc640eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7efc6ad9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc640eb290 0x7efc640eb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55380/0 (socket says 172.21.15.97:55380) 2024-03-29T04:07:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7efc6ad9d640 1 -- 172.21.15.97:0/1751358525 learned_addr learned my addr 172.21.15.97:0/1751358525 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7efc69d9b640 1 --2- 172.21.15.97:0/1751358525 >> v2:172.21.15.97:3300/0 conn(0x7efc64048c30 0x7efc64049050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7efc6ad9d640 1 -- 172.21.15.97:0/1751358525 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc64009180 msgr2=0x7efc64052ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:07:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7efc6ad9d640 1 --2- 172.21.15.97:0/1751358525 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc64009180 0x7efc64052ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7efc6ad9d640 1 -- 172.21.15.97:0/1751358525 >> v2:172.21.15.97:3300/0 conn(0x7efc64048c30 msgr2=0x7efc64049050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:45.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7efc6ad9d640 1 --2- 172.21.15.97:0/1751358525 >> v2:172.21.15.97:3300/0 conn(0x7efc64048c30 0x7efc64049050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7efc6ad9d640 1 -- 172.21.15.97:0/1751358525 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc64035c20 con 0x7efc640eb290 2024-03-29T04:07:45.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7fdc95bcd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc9015e920 0x7fdc9006c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7fdc95bcd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc900cf170 con 0x7fdc900cfbf0 2024-03-29T04:07:45.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7fdc95bcd640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc900d2270 con 0x7fdc9006cc60 2024-03-29T04:07:45.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7fdc95bcd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdc900d2510 con 0x7fdc9015e920 2024-03-29T04:07:45.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7f79be70e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b8163920 con 0x7f79b80d0490 2024-03-29T04:07:45.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7f79be70e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f79b80b3ee0 con 0x7f79b815d140 2024-03-29T04:07:45.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7f7af709a640 1 Processor -- start 2024-03-29T04:07:45.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7f7af709a640 1 -- start start 2024-03-29T04:07:45.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7fdc8f7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc9015e920 0x7fdc9006c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7fdc8f7fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc9015e920 0x7fdc9006c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:35568/0 (socket says 172.21.15.97:35568) 2024-03-29T04:07:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fdc8f7fe640 1 -- 172.21.15.97:0/2752858139 learned_addr learned my addr 172.21.15.97:0/2752858139 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.907+0000 7fc753fff640 1 -- start start 2024-03-29T04:07:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7fcb5b51b640 1 -- 172.21.15.97:0/2588919030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb40004ba0 msgr2=0x7fcb40004fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7fcb5b51b640 1 --2- 172.21.15.97:0/2588919030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb40004ba0 0x7fcb40004fa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcb500019c0 tx=0x7fcb40005830 comp rx=0 tx=0).stop 2024-03-29T04:07:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7fcb5b51b640 1 -- 172.21.15.97:0/2588919030 shutdown_connections 2024-03-29T04:07:45.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7fcb5b51b640 1 --2- 172.21.15.97:0/2588919030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb40004ba0 0x7fcb40004fa0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcb500019c0 tx=0x7fcb40005830 comp rx=0 tx=0).stop 2024-03-29T04:07:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.908+0000 7fcb5b51b640 1 -- 172.21.15.97:0/2588919030 >> 172.21.15.97:0/2588919030 conn(0x7fcb5408bfc0 msgr2=0x7fcb540d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7efc69d9b640 1 --2- 172.21.15.97:0/1751358525 >> v2:172.21.15.97:3300/0 conn(0x7efc64048c30 0x7efc64049050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f79b7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f79b815d140 0x7f79b806f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fc753fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc754166000 0x7fc754072670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fc753fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc754072bb0 0x7fc7540ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fc753fff640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7540718b0 con 0x7fc7540b5030 2024-03-29T04:07:45.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fc753fff640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fc754071660 con 0x7fc754166000 2024-03-29T04:07:45.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fc753fff640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc754071b90 con 0x7fc754072bb0 2024-03-29T04:07:45.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7af709a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7af00b38d0 0x7f7af016a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7af709a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7af00cf200 0x7f7af016ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7af709a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af00cd150 con 0x7f7af014c7b0 2024-03-29T04:07:45.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7af709a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f7af00cd430 con 0x7f7af00cf200 2024-03-29T04:07:45.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7af709a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f7af00cd710 con 0x7f7af00b38d0 2024-03-29T04:07:45.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7af4e36640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7af014c7b0 0x7f7af00b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:40416/0 (socket says 172.21.15.97:40416) 2024-03-29T04:07:45.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7af4e36640 1 -- 172.21.15.97:0/1118127904 learned_addr learned my addr 172.21.15.97:0/1118127904 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:45.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fdc8e7fc640 1 -- 172.21.15.97:0/2752858139 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1713388760 0 0) 0x7fdc900d2270 con 0x7fdc9006cc60 2024-03-29T04:07:45.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fdc8e7fc640 1 -- 172.21.15.97:0/2752858139 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc78003520 con 0x7fdc9006cc60 2024-03-29T04:07:45.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fdc8e7fc640 1 -- 172.21.15.97:0/2752858139 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1790377449 0 0) 0x7fdc88001490 con 0x7fdc900cfbf0 2024-03-29T04:07:45.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fdc8e7fc640 1 -- 172.21.15.97:0/2752858139 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc780048e0 con 0x7fdc900cfbf0 2024-03-29T04:07:45.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f49b96b0640 1 --2- 172.21.15.97:0/1432966571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49b406c720 0x7f49b40d4f40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f49b408e830 tx=0x7f49a402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f91152ebe04c83 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7aeffff640 1 --2- 172.21.15.97:0/1118127904 >> v2:172.21.15.154:3300/0 conn(0x7f7af00b38d0 0x7f7af016a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f79bccab640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f79b80d0490 0x7f79b8166bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:40414/0 (socket says 172.21.15.97:40414) 2024-03-29T04:07:45.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fc752ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc754166000 0x7fc754072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fc752ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc754166000 0x7fc754072670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38808/0 (socket says 172.21.15.97:38808) 2024-03-29T04:07:45.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fc752ffd640 1 -- 172.21.15.97:0/1858296907 learned_addr learned my addr 172.21.15.97:0/1858296907 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7af5637640 1 --2- 172.21.15.97:0/1118127904 >> v2:172.21.15.97:3300/0 conn(0x7f7af00cf200 0x7f7af016ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7aef7fe640 1 -- 172.21.15.97:0/1118127904 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3224674543 0 0) 0x7f7af00cd150 con 0x7f7af014c7b0 2024-03-29T04:07:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f79bccab640 1 -- 172.21.15.97:0/1078792976 learned_addr learned my addr 172.21.15.97:0/1078792976 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fdc8f7fe640 1 -- 172.21.15.97:0/2752858139 >> v1:172.21.15.154:6789/0 conn(0x7fdc9006cc60 legacy=0x7fdc900d4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fdc8f7fe640 1 -- 172.21.15.97:0/2752858139 >> v1:172.21.15.97:6789/0 conn(0x7fdc900cfbf0 legacy=0x7fdc900d2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fdc8f7fe640 1 -- 172.21.15.97:0/2752858139 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc900d27f0 con 0x7fdc9015e920 2024-03-29T04:07:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7fc752ffd640 1 -- 172.21.15.97:0/1858296907 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc754072bb0 msgr2=0x7fc7540ad1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:07:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7fc752ffd640 1 --2- 172.21.15.97:0/1858296907 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc754072bb0 0x7fc7540ad1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7fc752ffd640 1 -- 172.21.15.97:0/1858296907 >> v1:172.21.15.97:6789/0 conn(0x7fc7540b5030 legacy=0x7fc7540d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7fc752ffd640 1 -- 172.21.15.97:0/1858296907 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc754071e70 con 0x7fc754166000 2024-03-29T04:07:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7aef7fe640 1 -- 172.21.15.97:0/1118127904 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ad8003520 con 0x7f7af014c7b0 2024-03-29T04:07:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7aeffff640 1 -- 172.21.15.97:0/1118127904 >> v2:172.21.15.97:3300/0 conn(0x7f7af00cf200 msgr2=0x7f7af016ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7aeffff640 1 --2- 172.21.15.97:0/1118127904 >> v2:172.21.15.97:3300/0 conn(0x7f7af00cf200 0x7f7af016ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7aeffff640 1 -- 172.21.15.97:0/1118127904 >> v1:172.21.15.154:6789/0 conn(0x7f7af014c7b0 legacy=0x7f7af00b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7f7aeffff640 1 -- 172.21.15.97:0/1118127904 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af00cd9f0 con 0x7f7af00b38d0 2024-03-29T04:07:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.909+0000 7fcb5b51b640 1 -- 172.21.15.97:0/2588919030 shutdown_connections 2024-03-29T04:07:45.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7f79b7fff640 1 -- 172.21.15.97:0/1078792976 >> v1:172.21.15.154:6789/0 conn(0x7f79b80d0490 legacy=0x7f79b8166bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7efc6ad9d640 1 --2- 172.21.15.97:0/1751358525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc640eb290 0x7efc640eb6a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc6c02d940 tx=0x7efc6c0b8240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a3721962a21d0ad server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7fcb5b51b640 1 -- 172.21.15.97:0/2588919030 wait complete. 2024-03-29T04:07:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7f79b7fff640 1 -- 172.21.15.97:0/1078792976 >> v1:172.21.15.97:6789/0 conn(0x7f79b8163180 legacy=0x7f79b80b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7f79b7fff640 1 -- 172.21.15.97:0/1078792976 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b8163c00 con 0x7f79b815d140 2024-03-29T04:07:45.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7f49b8eaf640 1 -- 172.21.15.97:0/1432966571 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40317c0 con 0x7f49b406c720 2024-03-29T04:07:45.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7f49b8eaf640 1 -- 172.21.15.97:0/1432966571 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a4037070 con 0x7f49b406c720 2024-03-29T04:07:45.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7f49b8eaf640 1 -- 172.21.15.97:0/1432966571 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4031bc0 con 0x7f49b406c720 2024-03-29T04:07:45.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7efc6959a640 1 -- 172.21.15.97:0/1751358525 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0c5070 con 0x7efc640eb290 2024-03-29T04:07:45.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.910+0000 7efc6959a640 1 -- 172.21.15.97:0/1751358525 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc6c0c0680 con 0x7efc640eb290 2024-03-29T04:07:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7f7aeffff640 1 --2- 172.21.15.97:0/1118127904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7af00b38d0 0x7f7af016a760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7af01666a0 tx=0x7f7ae402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13332de7bd495269 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7fdc8f7fe640 1 --2- 172.21.15.97:0/2752858139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc9015e920 0x7fdc9006c720 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdc800098a0 tx=0x7fdc8002ba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f8cdf0f74df340f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7f7aef7fe640 1 -- 172.21.15.97:0/1118127904 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae40315a0 con 0x7f7af00b38d0 2024-03-29T04:07:45.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7f7aef7fe640 1 -- 172.21.15.97:0/1118127904 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae4037070 con 0x7f7af00b38d0 2024-03-29T04:07:45.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7fc752ffd640 1 --2- 172.21.15.97:0/1858296907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc754166000 0x7fc754072670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc744009870 tx=0x7fc74402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eae27e9fcc6b5f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7fdc8e7fc640 1 -- 172.21.15.97:0/2752858139 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8002f850 con 0x7fdc9015e920 2024-03-29T04:07:45.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7fdc8e7fc640 1 -- 172.21.15.97:0/2752858139 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc8002f9f0 con 0x7fdc9015e920 2024-03-29T04:07:45.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7fdc8e7fc640 1 -- 172.21.15.97:0/2752858139 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8002fe00 con 0x7fdc9015e920 2024-03-29T04:07:45.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7efc70b15640 1 -- 172.21.15.97:0/1751358525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc640eb290 msgr2=0x7efc640eb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7efc6959a640 1 -- 172.21.15.97:0/1751358525 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0c5070 con 0x7efc640eb290 2024-03-29T04:07:45.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7efc70b15640 1 --2- 172.21.15.97:0/1751358525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc640eb290 0x7efc640eb6a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc6c02d940 tx=0x7efc6c0b8240 comp rx=0 tx=0).stop 2024-03-29T04:07:45.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7f79b7fff640 1 --2- 172.21.15.97:0/1078792976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b815d140 0x7f79b806f0b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009d90 tx=0x7f79a802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c3c6776327f50d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7f7aef7fe640 1 -- 172.21.15.97:0/1118127904 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4031960 con 0x7f7af00b38d0 2024-03-29T04:07:45.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7fc751ffb640 1 -- 172.21.15.97:0/1858296907 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74402e070 con 0x7fc754166000 2024-03-29T04:07:45.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7efc70b15640 1 -- 172.21.15.97:0/1751358525 shutdown_connections 2024-03-29T04:07:45.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7efc70b15640 1 --2- 172.21.15.97:0/1751358525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc640eb290 0x7efc640eb6a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7efc70b15640 1 --2- 172.21.15.97:0/1751358525 >> v2:172.21.15.97:3300/0 conn(0x7efc64048c30 0x7efc64049050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7fc751ffb640 1 -- 172.21.15.97:0/1858296907 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc744037070 con 0x7fc754166000 2024-03-29T04:07:45.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7efc70b15640 1 --2- 172.21.15.97:0/1751358525 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc64009180 0x7efc64052ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7efc70b15640 1 -- 172.21.15.97:0/1751358525 >> 172.21.15.97:0/1751358525 conn(0x7efc6401ef30 msgr2=0x7efc64055c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7fc751ffb640 1 -- 172.21.15.97:0/1858296907 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744031a00 con 0x7fc754166000 2024-03-29T04:07:45.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7fcb5b51b640 1 Processor -- start 2024-03-29T04:07:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.911+0000 7f79b6ffd640 1 -- 172.21.15.97:0/1078792976 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8039070 con 0x7f79b815d140 2024-03-29T04:07:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7efc70b15640 1 -- 172.21.15.97:0/1751358525 shutdown_connections 2024-03-29T04:07:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f79b6ffd640 1 -- 172.21.15.97:0/1078792976 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a802cc60 con 0x7f79b815d140 2024-03-29T04:07:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7fcb5b51b640 1 -- start start 2024-03-29T04:07:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f7af709a640 1 -- 172.21.15.97:0/1118127904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7af00b38d0 msgr2=0x7f7af016a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f7af709a640 1 --2- 172.21.15.97:0/1118127904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7af00b38d0 0x7f7af016a760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7af01666a0 tx=0x7f7ae402b840 comp rx=0 tx=0).stop 2024-03-29T04:07:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f7af709a640 1 -- 172.21.15.97:0/1118127904 shutdown_connections 2024-03-29T04:07:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f7af709a640 1 --2- 172.21.15.97:0/1118127904 >> v2:172.21.15.97:3300/0 conn(0x7f7af00cf200 0x7f7af016ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f7af709a640 1 --2- 172.21.15.97:0/1118127904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7af00b38d0 0x7f7af016a760 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7af01666a0 tx=0x7f7ae402b840 comp rx=0 tx=0).stop 2024-03-29T04:07:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f7af709a640 1 -- 172.21.15.97:0/1118127904 >> 172.21.15.97:0/1118127904 conn(0x7f7af00b56e0 msgr2=0x7f7af00b5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7efc70b15640 1 -- 172.21.15.97:0/1751358525 wait complete. 2024-03-29T04:07:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f49bc115640 1 -- 172.21.15.97:0/1432966571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49b406c720 msgr2=0x7f49b40d4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f49bc115640 1 --2- 172.21.15.97:0/1432966571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49b406c720 0x7f49b40d4f40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f49b408e830 tx=0x7f49a402b860 comp rx=0 tx=0).stop 2024-03-29T04:07:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f49bc115640 1 -- 172.21.15.97:0/1432966571 shutdown_connections 2024-03-29T04:07:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f49bc115640 1 --2- 172.21.15.97:0/1432966571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49b406c720 0x7f49b40d4f40 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f49b408e830 tx=0x7f49a402b860 comp rx=0 tx=0).stop 2024-03-29T04:07:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f49bc115640 1 --2- 172.21.15.97:0/1432966571 >> v2:172.21.15.154:3300/0 conn(0x7f49b40b3cc0 0x7f49b40d1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f49bc115640 1 -- 172.21.15.97:0/1432966571 >> 172.21.15.97:0/1432966571 conn(0x7f49b408bfc0 msgr2=0x7f49b40d46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7fdc95bcd640 1 -- 172.21.15.97:0/2752858139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc9015e920 msgr2=0x7fdc9006c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7fdc95bcd640 1 --2- 172.21.15.97:0/2752858139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc9015e920 0x7fdc9006c720 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdc800098a0 tx=0x7fdc8002ba30 comp rx=0 tx=0).stop 2024-03-29T04:07:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7fdc95bcd640 1 -- 172.21.15.97:0/2752858139 shutdown_connections 2024-03-29T04:07:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7fdc95bcd640 1 --2- 172.21.15.97:0/2752858139 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc9015e920 0x7fdc9006c720 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7fdc95bcd640 1 -- 172.21.15.97:0/2752858139 >> 172.21.15.97:0/2752858139 conn(0x7fdc9008bfc0 msgr2=0x7fdc900d46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7fdc95bcd640 1 -- 172.21.15.97:0/2752858139 shutdown_connections 2024-03-29T04:07:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7fc753fff640 1 -- 172.21.15.97:0/1858296907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc754166000 msgr2=0x7fc754072670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7fc753fff640 1 --2- 172.21.15.97:0/1858296907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc754166000 0x7fc754072670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc744009870 tx=0x7fc74402b880 comp rx=0 tx=0).stop 2024-03-29T04:07:45.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7fdc95bcd640 1 -- 172.21.15.97:0/2752858139 wait complete. 2024-03-29T04:07:45.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.912+0000 7f79b6ffd640 1 -- 172.21.15.97:0/1078792976 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8034660 con 0x7f79b815d140 2024-03-29T04:07:45.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7f79be70e640 1 -- 172.21.15.97:0/1078792976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b815d140 msgr2=0x7f79b806f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7f79be70e640 1 --2- 172.21.15.97:0/1078792976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b815d140 0x7f79b806f0b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009d90 tx=0x7f79a802beb0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7f49bc115640 1 -- 172.21.15.97:0/1432966571 shutdown_connections 2024-03-29T04:07:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7f49bc115640 1 -- 172.21.15.97:0/1432966571 wait complete. 2024-03-29T04:07:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7f7af709a640 1 -- 172.21.15.97:0/1118127904 shutdown_connections 2024-03-29T04:07:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fcb5b51b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb54158910 0x7fcb5415f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7f79be70e640 1 -- 172.21.15.97:0/1078792976 shutdown_connections 2024-03-29T04:07:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7f79be70e640 1 --2- 172.21.15.97:0/1078792976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b815d140 0x7f79b806f0b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7f79be70e640 1 -- 172.21.15.97:0/1078792976 >> 172.21.15.97:0/1078792976 conn(0x7f79b806b6f0 msgr2=0x7f79b806bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7f79be70e640 1 -- 172.21.15.97:0/1078792976 shutdown_connections 2024-03-29T04:07:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7f79be70e640 1 -- 172.21.15.97:0/1078792976 wait complete. 2024-03-29T04:07:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fcb592b7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb54158910 0x7fcb5415f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fcb592b7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb54158910 0x7fcb5415f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38814/0 (socket says 172.21.15.97:38814) 2024-03-29T04:07:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fcb592b7640 1 -- 172.21.15.97:0/3244344505 learned_addr learned my addr 172.21.15.97:0/3244344505 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fcb5b51b640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb541617b0 0x7fcb540b9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fcb5b51b640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb540b98a0 0x7fcb540b7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fc753fff640 1 -- 172.21.15.97:0/1858296907 shutdown_connections 2024-03-29T04:07:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fc753fff640 1 --2- 172.21.15.97:0/1858296907 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc754072bb0 0x7fc7540ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcb540d4ef0 con 0x7fcb54158910 2024-03-29T04:07:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fcb58ab6640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb541617b0 0x7fcb540b9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fc753fff640 1 --2- 172.21.15.97:0/1858296907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc754166000 0x7fc754072670 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fcb59ab8640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb540b98a0 0x7fcb540b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcb540d2d20 con 0x7fcb540b98a0 2024-03-29T04:07:45.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcb540d5070 con 0x7fcb541617b0 2024-03-29T04:07:45.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7fc753fff640 1 -- 172.21.15.97:0/1858296907 >> 172.21.15.97:0/1858296907 conn(0x7fc7540b22e0 msgr2=0x7fc7540b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.913+0000 7f7af709a640 1 -- 172.21.15.97:0/1118127904 wait complete. 2024-03-29T04:07:45.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7fcb58ab6640 1 -- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb540b98a0 msgr2=0x7fcb540b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7fcb58ab6640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb540b98a0 0x7fcb540b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7fcb58ab6640 1 -- 172.21.15.97:0/3244344505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb54158910 msgr2=0x7fcb5415f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7efc70b15640 1 Processor -- start 2024-03-29T04:07:45.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7fdc95bcd640 1 Processor -- start 2024-03-29T04:07:45.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7fcb58ab6640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb54158910 0x7fcb5415f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7fcb58ab6640 1 -- 172.21.15.97:0/3244344505 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb540b5260 con 0x7fcb541617b0 2024-03-29T04:07:45.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7efc70b15640 1 -- start start 2024-03-29T04:07:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7f49bc115640 1 Processor -- start 2024-03-29T04:07:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7fcb592b7640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb54158910 0x7fcb5415f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7fdc95bcd640 1 -- start start 2024-03-29T04:07:45.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7fc753fff640 1 -- 172.21.15.97:0/1858296907 shutdown_connections 2024-03-29T04:07:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7f49bc115640 1 -- start start 2024-03-29T04:07:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7fcb58ab6640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb541617b0 0x7fcb540b9360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcb44010790 tx=0x7fcb44010c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.914+0000 7f79be70e640 1 Processor -- start 2024-03-29T04:07:45.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7fcb49ffb640 1 -- 172.21.15.97:0/3244344505 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400f040 con 0x7fcb541617b0 2024-03-29T04:07:45.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7f79be70e640 1 -- start start 2024-03-29T04:07:45.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5415fdf0 con 0x7fcb541617b0 2024-03-29T04:07:45.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb54160350 con 0x7fcb541617b0 2024-03-29T04:07:45.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7fc753fff640 1 -- 172.21.15.97:0/1858296907 wait complete. 2024-03-29T04:07:45.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7f49bc115640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49b40b3cc0 0x7f49b40c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7f49bc115640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49b40d2c80 0x7f49b40cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7fcb49ffb640 1 -- 172.21.15.97:0/3244344505 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb4400e950 con 0x7fcb541617b0 2024-03-29T04:07:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7efc70b15640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc64009180 0x7efc64051be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7efc70b15640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc64048c30 0x7efc640d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7fcb49ffb640 1 -- 172.21.15.97:0/3244344505 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400ed00 con 0x7fcb541617b0 2024-03-29T04:07:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7f49bc115640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49b414c820 0x7f49b40cde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7f49bc115640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f49b40cf3f0 con 0x7f49b414c820 2024-03-29T04:07:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7f49bc115640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f49b40c7d20 con 0x7f49b40b3cc0 2024-03-29T04:07:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7f49bc115640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f49b40cf910 con 0x7f49b40d2c80 2024-03-29T04:07:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7f49b9eb1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49b40b3cc0 0x7f49b40c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7f49b9eb1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49b40b3cc0 0x7f49b40c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55424/0 (socket says 172.21.15.97:55424) 2024-03-29T04:07:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f49b96b0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49b40d2c80 0x7f49b40cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f49b9eb1640 1 -- 172.21.15.97:0/1301994387 learned_addr learned my addr 172.21.15.97:0/1301994387 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7efc6a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc64009180 0x7efc64051be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7efc70b15640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc640eb290 0x7efc64050290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7efc70b15640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efc640364c0 con 0x7efc64048c30 2024-03-29T04:07:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7efc70b15640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efc640361e0 con 0x7efc64009180 2024-03-29T04:07:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7efc70b15640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7efc640367a0 con 0x7efc640eb290 2024-03-29T04:07:45.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f49ba6b2640 1 --2- 172.21.15.97:0/1301994387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49b414c820 0x7f49b40cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7fdc95bcd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc9006cc60 0x7fdc900cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7fdc95bcd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc900cfbf0 0x7fdc901f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f79be70e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79b80d0490 0x7f79b8158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7efc69d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc64048c30 0x7efc640d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7efc69d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc64048c30 0x7efc640d8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38834/0 (socket says 172.21.15.97:38834) 2024-03-29T04:07:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7efc69d9b640 1 -- 172.21.15.97:0/2255786642 learned_addr learned my addr 172.21.15.97:0/2255786642 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f79be70e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b815d140 0x7f79b80b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f79be70e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b8163180 0x7f79b80cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f79be70e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f79b815d520 con 0x7f79b8163180 2024-03-29T04:07:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f79b77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b815d140 0x7f79b80b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f79b77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b815d140 0x7f79b80b0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55440/0 (socket says 172.21.15.97:55440) 2024-03-29T04:07:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f79b77fe640 1 -- 172.21.15.97:0/1982087663 learned_addr learned my addr 172.21.15.97:0/1982087663 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f79be70e640 1 -- 172.21.15.97:0/1982087663 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f79b80b33f0 con 0x7f79b815d140 2024-03-29T04:07:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f79be70e640 1 -- 172.21.15.97:0/1982087663 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f79b81672d0 con 0x7f79b80d0490 2024-03-29T04:07:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f49b96b0640 1 -- 172.21.15.97:0/1301994387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49b40b3cc0 msgr2=0x7f49b40c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f49b96b0640 1 --2- 172.21.15.97:0/1301994387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49b40b3cc0 0x7f49b40c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f49b96b0640 1 -- 172.21.15.97:0/1301994387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49b414c820 msgr2=0x7f49b40cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f49b96b0640 1 --2- 172.21.15.97:0/1301994387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49b414c820 0x7f49b40cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7f49b96b0640 1 -- 172.21.15.97:0/1301994387 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b40c8340 con 0x7f49b40d2c80 2024-03-29T04:07:45.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f79b7fff640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79b80d0490 0x7f79b8158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7efc69d9b640 1 -- 172.21.15.97:0/2255786642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc640eb290 msgr2=0x7efc64050290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:07:45.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7efc69d9b640 1 --2- 172.21.15.97:0/2255786642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc640eb290 0x7efc64050290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7efc69d9b640 1 -- 172.21.15.97:0/2255786642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc64009180 msgr2=0x7efc64051be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7efc69d9b640 1 --2- 172.21.15.97:0/2255786642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc64009180 0x7efc64051be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.916+0000 7efc69d9b640 1 -- 172.21.15.97:0/2255786642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc640e72b0 con 0x7efc64048c30 2024-03-29T04:07:45.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7fc753fff640 1 Processor -- start 2024-03-29T04:07:45.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f49b96b0640 1 --2- 172.21.15.97:0/1301994387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49b40d2c80 0x7f49b40cd940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f49a4033000 tx=0x7f49a40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7f7af709a640 1 Processor -- start 2024-03-29T04:07:45.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7fb9f759e640 1 Processor -- start 2024-03-29T04:07:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f49aa7fc640 1 -- 172.21.15.97:0/1301994387 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4031e40 con 0x7f49b40d2c80 2024-03-29T04:07:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.915+0000 7f7af709a640 1 -- start start 2024-03-29T04:07:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7fc753fff640 1 -- start start 2024-03-29T04:07:45.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f79bccab640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b8163180 0x7f79b80cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f79b77fe640 1 -- 172.21.15.97:0/1982087663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79b80d0490 msgr2=0x7f79b8158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f79b77fe640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79b80d0490 0x7f79b8158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f79b77fe640 1 -- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b8163180 msgr2=0x7f79b80cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f79b77fe640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b8163180 0x7f79b80cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f79b77fe640 1 -- 172.21.15.97:0/1982087663 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b81ec890 con 0x7f79b815d140 2024-03-29T04:07:45.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f79bccab640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b8163180 0x7f79b80cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:07:45.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f79b77fe640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b815d140 0x7f79b80b0c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f79b000c3c0 tx=0x7f79b000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f49aa7fc640 1 -- 172.21.15.97:0/1301994387 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a4004030 con 0x7f49b40d2c80 2024-03-29T04:07:45.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7efc69d9b640 1 --2- 172.21.15.97:0/2255786642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc64048c30 0x7efc640d8e20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efc6000e7e0 tx=0x7efc6000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7efc5affd640 1 -- 172.21.15.97:0/2255786642 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6000f040 con 0x7efc64048c30 2024-03-29T04:07:45.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7efc70b15640 1 -- 172.21.15.97:0/2255786642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc640e7590 con 0x7efc64048c30 2024-03-29T04:07:45.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af709a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af00cf200 0x7f7af01f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af709a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7af014c7b0 0x7f7af00947d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af709a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7af0094d10 0x7f7af009d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af709a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7af00cff10 con 0x7f7af00cf200 2024-03-29T04:07:45.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af709a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7af00d0fb0 con 0x7f7af014c7b0 2024-03-29T04:07:45.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af709a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7af00d0090 con 0x7f7af0094d10 2024-03-29T04:07:45.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7aeffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7af014c7b0 0x7f7af00947d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7aeffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7af014c7b0 0x7f7af00947d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55446/0 (socket says 172.21.15.97:55446) 2024-03-29T04:07:45.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7aeffff640 1 -- 172.21.15.97:0/2913553956 learned_addr learned my addr 172.21.15.97:0/2913553956 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af4e36640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af00cf200 0x7f7af01f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af5637640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7af0094d10 0x7f7af009d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af4e36640 1 -- 172.21.15.97:0/2913553956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7af0094d10 msgr2=0x7f7af009d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af4e36640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7af0094d10 0x7f7af009d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af4e36640 1 -- 172.21.15.97:0/2913553956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7af014c7b0 msgr2=0x7f7af00947d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af4e36640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7af014c7b0 0x7f7af00947d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af4e36640 1 -- 172.21.15.97:0/2913553956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af0158970 con 0x7f7af00cf200 2024-03-29T04:07:45.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7f7af4e36640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af00cf200 0x7f7af01f7b10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000c430 tx=0x7f7ae000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7fdc95bcd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9015e920 0x7fdc901f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7fb9f759e640 1 -- start start 2024-03-29T04:07:45.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.917+0000 7fcb49ffb640 1 -- 172.21.15.97:0/3244344505 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcb440143f0 con 0x7fcb541617b0 2024-03-29T04:07:45.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcb34001030 con 0x7fcb541617b0 2024-03-29T04:07:45.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7fdc8f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc9006cc60 0x7fdc900cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7fdc8f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc9006cc60 0x7fdc900cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55454/0 (socket says 172.21.15.97:55454) 2024-03-29T04:07:45.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7efc70b15640 1 -- 172.21.15.97:0/2255786642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc640dd5e0 con 0x7efc64048c30 2024-03-29T04:07:45.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7fdc8f7fe640 1 -- 172.21.15.97:0/3264736381 learned_addr learned my addr 172.21.15.97:0/3264736381 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdc900d54c0 con 0x7fdc9015e920 2024-03-29T04:07:45.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdc900cf6d0 con 0x7fdc9006cc60 2024-03-29T04:07:45.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdc900d3350 con 0x7fdc900cfbf0 2024-03-29T04:07:45.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7f79b4ff9640 1 -- 172.21.15.97:0/1982087663 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b0017070 con 0x7f79b815d140 2024-03-29T04:07:45.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7fdc8effd640 1 --2- 172.21.15.97:0/3264736381 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc900cfbf0 0x7fdc901f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7fcb49ffb640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcb3806c630 0x7fcb3806eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7f79b4ff9640 1 -- 172.21.15.97:0/1982087663 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b00052d0 con 0x7f79b815d140 2024-03-29T04:07:45.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7f79b4ff9640 1 -- 172.21.15.97:0/1982087663 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b0003830 con 0x7f79b815d140 2024-03-29T04:07:45.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7f7aed7fa640 1 -- 172.21.15.97:0/2913553956 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0017070 con 0x7f7af00cf200 2024-03-29T04:07:45.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7f79be70e640 1 -- 172.21.15.97:0/1982087663 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b81ecb70 con 0x7f79b815d140 2024-03-29T04:07:45.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7f7aed7fa640 1 -- 172.21.15.97:0/2913553956 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae0005ce0 con 0x7f7af00cf200 2024-03-29T04:07:45.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fc753fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc754072bb0 0x7fc75409bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fc753fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7540b5030 0x7fc7540ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7f79be70e640 1 -- 172.21.15.97:0/1982087663 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b80b13c0 con 0x7f79b815d140 2024-03-29T04:07:45.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7f49bc115640 1 -- 172.21.15.97:0/1301994387 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b40c8620 con 0x7f49b40d2c80 2024-03-29T04:07:45.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7f49bc115640 1 -- 172.21.15.97:0/1301994387 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b40c8be0 con 0x7f49b40d2c80 2024-03-29T04:07:45.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7f49aa7fc640 1 -- 172.21.15.97:0/1301994387 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4004420 con 0x7f49b40d2c80 2024-03-29T04:07:45.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fb9f759e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9f80cfad0 0x7fb9f80d4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fb9f759e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb9f80d0ce0 0x7fb9f80d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fb9f659c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb9f815e570 0x7fb9f80d2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:40418/0 (socket says 172.21.15.97:40418) 2024-03-29T04:07:45.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fb9f659c640 1 -- 172.21.15.97:0/2996283682 learned_addr learned my addr 172.21.15.97:0/2996283682 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:45.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7fcb592b7640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcb3806c630 0x7fcb3806eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fc753fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc754166000 0x7fc75409a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fb9f759e640 1 -- 172.21.15.97:0/2996283682 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9f80d01a0 con 0x7fb9f815e570 2024-03-29T04:07:45.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fc753fff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc754073910 con 0x7fc754072bb0 2024-03-29T04:07:45.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fc753fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc754073790 con 0x7fc7540b5030 2024-03-29T04:07:45.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fb9f759e640 1 -- 172.21.15.97:0/2996283682 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fb9f80cff60 con 0x7fb9f80d0ce0 2024-03-29T04:07:45.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fc753fff640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc7540d0c60 con 0x7fc754166000 2024-03-29T04:07:45.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fb9f759e640 1 -- 172.21.15.97:0/2996283682 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f80d53e0 con 0x7fb9f80cfad0 2024-03-29T04:07:45.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7f7aed7fa640 1 -- 172.21.15.97:0/2913553956 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae00072d0 con 0x7f7af00cf200 2024-03-29T04:07:45.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fcb49ffb640 1 -- 172.21.15.97:0/3244344505 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcb4408bcc0 con 0x7fcb541617b0 2024-03-29T04:07:45.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fc7527fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7540b5030 0x7fc7540ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fc7527fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7540b5030 0x7fc7540ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55460/0 (socket says 172.21.15.97:55460) 2024-03-29T04:07:45.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2128503014 0 0) 0x7fb9f80d01a0 con 0x7fb9f815e570 2024-03-29T04:07:45.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9e0003520 con 0x7fb9f815e570 2024-03-29T04:07:45.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fc7527fc640 1 -- 172.21.15.97:0/1270699321 learned_addr learned my addr 172.21.15.97:0/1270699321 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fcb49ffb640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] conn(0x7fcb38074850 0x7fcb38076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fb9f6d9d640 1 --2- 172.21.15.97:0/2996283682 >> v2:172.21.15.154:3300/0 conn(0x7fb9f80d0ce0 0x7fb9f80d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fdc8f7fe640 1 -- 172.21.15.97:0/3264736381 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc900cfbf0 msgr2=0x7fdc901f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fb9f5d9b640 1 --2- 172.21.15.97:0/2996283682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9f80cfad0 0x7fb9f80d4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fdc8f7fe640 1 --2- 172.21.15.97:0/3264736381 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdc900cfbf0 0x7fdc901f5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fdc8f7fe640 1 -- 172.21.15.97:0/3264736381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9015e920 msgr2=0x7fdc901f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fdc8f7fe640 1 --2- 172.21.15.97:0/3264736381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc9015e920 0x7fdc901f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fdc8f7fe640 1 -- 172.21.15.97:0/3264736381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc900c83e0 con 0x7fdc9006cc60 2024-03-29T04:07:45.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fcb49ffb640 1 -- 172.21.15.97:0/3244344505 --> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcb38077320 con 0x7fcb38074850 2024-03-29T04:07:45.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fcb49ffb640 1 -- 172.21.15.97:0/3244344505 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcb44091260 con 0x7fcb541617b0 2024-03-29T04:07:45.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.919+0000 7fcb59ab8640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] conn(0x7fcb38074850 0x7fcb38076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fdc8f7fe640 1 --2- 172.21.15.97:0/3264736381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc9006cc60 0x7fdc900cd960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdc800098a0 tx=0x7fdc800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3382329325 0 0) 0x7fb9e0003520 con 0x7fb9f815e570 2024-03-29T04:07:45.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fcb592b7640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcb3806c630 0x7fcb3806eaf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000c00 tx=0x7fcb50009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7efc5affd640 1 -- 172.21.15.97:0/2255786642 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc60004030 con 0x7efc64048c30 2024-03-29T04:07:45.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.918+0000 7efc5affd640 1 -- 172.21.15.97:0/2255786642 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc600043e0 con 0x7efc64048c30 2024-03-29T04:07:45.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9f80cfad0 msgr2=0x7fb9f80d4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:45.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 --2- 172.21.15.97:0/2996283682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9f80cfad0 0x7fb9f80d4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 >> v2:172.21.15.154:3300/0 conn(0x7fb9f80d0ce0 msgr2=0x7fb9f80d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:45.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 --2- 172.21.15.97:0/2996283682 >> v2:172.21.15.154:3300/0 conn(0x7fb9f80d0ce0 0x7fb9f80d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f80d21f0 con 0x7fb9f815e570 2024-03-29T04:07:45.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7fb9e8001670 con 0x7fb9f815e570 2024-03-29T04:07:45.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 >> v1:172.21.15.154:6789/0 conn(0x7fb9f815e570 legacy=0x7fb9f80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 --2- 172.21.15.97:0/2996283682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e00039b0 0x7fb9e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fc7537fe640 1 --2- 172.21.15.97:0/1270699321 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc754166000 0x7fc75409a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 --2- 172.21.15.97:0/2996283682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e0004800 0x7fb9e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 --2- 172.21.15.97:0/2996283682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9f80cfad0 0x7fb9e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f80cff60 con 0x7fb9e00039b0 2024-03-29T04:07:45.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f80d53e0 con 0x7fb9e0004800 2024-03-29T04:07:45.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e000ba40 con 0x7fb9f80cfad0 2024-03-29T04:07:45.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7f7af709a640 1 -- 172.21.15.97:0/2913553956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af0095340 con 0x7f7af00cf200 2024-03-29T04:07:45.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7f7af709a640 1 -- 172.21.15.97:0/2913553956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af0096350 con 0x7f7af00cf200 2024-03-29T04:07:45.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fc7537fe640 1 -- 172.21.15.97:0/1270699321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7540b5030 msgr2=0x7fc7540ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fc7537fe640 1 --2- 172.21.15.97:0/1270699321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7540b5030 0x7fc7540ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fc7537fe640 1 -- 172.21.15.97:0/1270699321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc754072bb0 msgr2=0x7fc75409bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fc7537fe640 1 --2- 172.21.15.97:0/1270699321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc754072bb0 0x7fc75409bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fc7537fe640 1 -- 172.21.15.97:0/1270699321 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75414e7e0 con 0x7fc754166000 2024-03-29T04:07:45.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fcb59ab8640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] conn(0x7fcb38074850 0x7fcb38076c70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f5d9b640 1 --2- 172.21.15.97:0/2996283682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e0004800 0x7fb9e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f659c640 1 --2- 172.21.15.97:0/2996283682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e00039b0 0x7fb9e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.920+0000 7fb9f6d9d640 1 --2- 172.21.15.97:0/2996283682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9f80cfad0 0x7fb9e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7fdc77fff640 1 -- 172.21.15.97:0/3264736381 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80037c40 con 0x7fdc9006cc60 2024-03-29T04:07:45.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7f79be70e640 1 -- 172.21.15.97:0/1982087663 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f799c001030 con 0x7f79b815d140 2024-03-29T04:07:45.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7f49bc115640 1 -- 172.21.15.97:0/1301994387 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49b40c8f20 con 0x7f49b40d2c80 2024-03-29T04:07:45.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc900c86c0 con 0x7fdc9006cc60 2024-03-29T04:07:45.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7f49aa7fc640 1 -- 172.21.15.97:0/1301994387 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f49a4003610 con 0x7f49b40d2c80 2024-03-29T04:07:45.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7fb9f5d9b640 1 -- 172.21.15.97:0/2996283682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9f80cfad0 msgr2=0x7fb9e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:45.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7fb9f5d9b640 1 --2- 172.21.15.97:0/2996283682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9f80cfad0 0x7fb9e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7f79b4ff9640 1 -- 172.21.15.97:0/1982087663 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f79b001a070 con 0x7f79b815d140 2024-03-29T04:07:45.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc900c8c80 con 0x7fdc9006cc60 2024-03-29T04:07:45.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7fb9f5d9b640 1 -- 172.21.15.97:0/2996283682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e00039b0 msgr2=0x7fb9e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:45.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7fb9f5d9b640 1 --2- 172.21.15.97:0/2996283682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e00039b0 0x7fb9e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7fb9f5d9b640 1 -- 172.21.15.97:0/2996283682 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e0003520 con 0x7fb9e0004800 2024-03-29T04:07:45.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7fdc77fff640 1 -- 172.21.15.97:0/3264736381 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc80047070 con 0x7fdc9006cc60 2024-03-29T04:07:45.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7fdc77fff640 1 -- 172.21.15.97:0/3264736381 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80042480 con 0x7fdc9006cc60 2024-03-29T04:07:45.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7f49aa7fc640 1 --2- 172.21.15.97:0/1301994387 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f499806c580 0x7f499806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7fc7537fe640 1 --2- 172.21.15.97:0/1270699321 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc754166000 0x7fc75409a2b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00c780 tx=0x7fc74c00cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f7af709a640 1 -- 172.21.15.97:0/2913553956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7af0068640 con 0x7f7af00cf200 2024-03-29T04:07:45.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f79b4ff9640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f799806c600 0x7f799806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f49aa7fc640 1 -- 172.21.15.97:0/1301994387 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f49a40ad080 con 0x7f49b40d2c80 2024-03-29T04:07:45.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7fc73b7fe640 1 -- 172.21.15.97:0/1270699321 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c016020 con 0x7fc754166000 2024-03-29T04:07:45.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f79b4ff9640 1 -- 172.21.15.97:0/1982087663 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f79b008a930 con 0x7f79b815d140 2024-03-29T04:07:45.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f79b4ff9640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] conn(0x7f79980747a0 0x7f7998076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f49aa7fc640 1 --2- 172.21.15.97:0/1301994387 >> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] conn(0x7f4998074780 0x7f4998076ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7fc753fff640 1 -- 172.21.15.97:0/1270699321 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75414eac0 con 0x7fc754166000 2024-03-29T04:07:45.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f79b4ff9640 1 -- 172.21.15.97:0/1982087663 --> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7998077270 con 0x7f79980747a0 2024-03-29T04:07:45.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f79b4ff9640 1 -- 172.21.15.97:0/1982087663 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f79b0054b60 con 0x7f79b815d140 2024-03-29T04:07:45.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7fc753fff640 1 -- 172.21.15.97:0/1270699321 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75414f080 con 0x7fc754166000 2024-03-29T04:07:45.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7fc73b7fe640 1 -- 172.21.15.97:0/1270699321 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc74c00aa50 con 0x7fc754166000 2024-03-29T04:07:45.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7fc73b7fe640 1 -- 172.21.15.97:0/1270699321 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00ae40 con 0x7fc754166000 2024-03-29T04:07:45.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f79b7fff640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f799806c600 0x7f799806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f79bccab640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] conn(0x7f79980747a0 0x7f7998076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f49ba6b2640 1 --2- 172.21.15.97:0/1301994387 >> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] conn(0x7f4998074780 0x7f4998076ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f49aa7fc640 1 -- 172.21.15.97:0/1301994387 --> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49a4004030 con 0x7f4998074780 2024-03-29T04:07:45.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f49aa7fc640 1 -- 172.21.15.97:0/1301994387 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f49a40b4050 con 0x7f49b40d2c80 2024-03-29T04:07:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.921+0000 7efc70b15640 1 -- 172.21.15.97:0/2255786642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efc640e7060 con 0x7efc64048c30 2024-03-29T04:07:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7fb9f6d9d640 1 --2- 172.21.15.97:0/2996283682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9f80cfad0 0x7fb9e00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7fcb49ffb640 1 -- 172.21.15.97:0/3244344505 <== osd.7 v2:172.21.15.154:6816/2375296016 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcb38077320 con 0x7fcb38074850 2024-03-29T04:07:45.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.922+0000 7f49b9eb1640 1 --2- 172.21.15.97:0/1301994387 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f499806c580 0x7f499806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.923+0000 7f79bccab640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] conn(0x7f79980747a0 0x7f7998076bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.923+0000 7f49ba6b2640 1 --2- 172.21.15.97:0/1301994387 >> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] conn(0x7f4998074780 0x7f4998076ba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.923+0000 7f49b9eb1640 1 --2- 172.21.15.97:0/1301994387 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f499806c580 0x7f499806ea40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009c00 tx=0x7f49b0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.923+0000 7efc5affd640 1 -- 172.21.15.97:0/2255786642 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efc6000c6d0 con 0x7efc64048c30 2024-03-29T04:07:45.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.923+0000 7fdc77fff640 1 -- 172.21.15.97:0/3264736381 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdc80040050 con 0x7fdc9006cc60 2024-03-29T04:07:45.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.924+0000 7f49aa7fc640 1 -- 172.21.15.97:0/1301994387 <== osd.2 v2:172.21.15.97:6816/127556154 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49a4004030 con 0x7f4998074780 2024-03-29T04:07:45.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.924+0000 7efc5affd640 1 --2- 172.21.15.97:0/2255786642 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7efc4c06c600 0x7efc4c06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.924+0000 7f7aed7fa640 1 -- 172.21.15.97:0/2913553956 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7ae001a070 con 0x7f7af00cf200 2024-03-29T04:07:45.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.924+0000 7fdc77fff640 1 --2- 172.21.15.97:0/3264736381 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdc7c06c600 0x7fdc7c06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.924+0000 7efc6a59c640 1 --2- 172.21.15.97:0/2255786642 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7efc4c06c600 0x7efc4c06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.924+0000 7efc5affd640 1 -- 172.21.15.97:0/2255786642 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efc6008b400 con 0x7efc64048c30 2024-03-29T04:07:45.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.924+0000 7fdc8effd640 1 --2- 172.21.15.97:0/3264736381 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdc7c06c600 0x7fdc7c06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.924+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc70001030 con 0x7fdc9006cc60 2024-03-29T04:07:45.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.924+0000 7f79b7fff640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f799806c600 0x7f799806eac0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f79a8036000 tx=0x7f79a8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.924+0000 7f79b4ff9640 1 -- 172.21.15.97:0/1982087663 <== osd.1 v2:172.21.15.97:6808/4164988844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7998077270 con 0x7f79980747a0 2024-03-29T04:07:45.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.924+0000 7efc5affd640 1 --2- 172.21.15.97:0/2255786642 >> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] conn(0x7efc4c0747a0 0x7efc4c076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.924+0000 7efc5affd640 1 -- 172.21.15.97:0/2255786642 --> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efc60022e50 con 0x7efc4c0747a0 2024-03-29T04:07:45.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7efc5affd640 1 -- 172.21.15.97:0/2255786642 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7efc60092050 con 0x7efc64048c30 2024-03-29T04:07:45.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7f7aed7fa640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7adc06c600 0x7f7adc06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7fc73b7fe640 1 -- 172.21.15.97:0/1270699321 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc74c019020 con 0x7fc754166000 2024-03-29T04:07:45.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7fc753fff640 1 -- 172.21.15.97:0/1270699321 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc734001030 con 0x7fc754166000 2024-03-29T04:07:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7efc6ad9d640 1 --2- 172.21.15.97:0/2255786642 >> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] conn(0x7efc4c0747a0 0x7efc4c076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7fdc77fff640 1 -- 172.21.15.97:0/3264736381 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdc800ae080 con 0x7fdc9006cc60 2024-03-29T04:07:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7fdc77fff640 1 --2- 172.21.15.97:0/3264736381 >> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] conn(0x7fdc7c074800 0x7fdc7c076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7fdc77fff640 1 -- 172.21.15.97:0/3264736381 --> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc80034e70 con 0x7fdc7c074800 2024-03-29T04:07:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7fdc77fff640 1 -- 172.21.15.97:0/3264736381 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdc8007bde0 con 0x7fdc9006cc60 2024-03-29T04:07:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.924+0000 7fb9f5d9b640 1 --2- 172.21.15.97:0/2996283682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e0004800 0x7fb9e0004c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000e170 tx=0x7fb9e0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a0fed9f2f33aafe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0039070 con 0x7fb9e0004800 2024-03-29T04:07:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9f002cc00 con 0x7fb9e0004800 2024-03-29T04:07:45.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7fb9f559a640 1 -- 172.21.15.97:0/2996283682 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0009c10 con 0x7fb9e0004800 2024-03-29T04:07:45.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7fdc8ffff640 1 --2- 172.21.15.97:0/3264736381 >> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] conn(0x7fdc7c074800 0x7fdc7c076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7efc6ad9d640 1 --2- 172.21.15.97:0/2255786642 >> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] conn(0x7efc4c0747a0 0x7efc4c076bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7efc6a59c640 1 --2- 172.21.15.97:0/2255786642 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7efc4c06c600 0x7efc4c06eac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc5c006fd0 tx=0x7efc5c009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7fc73b7fe640 1 --2- 172.21.15.97:0/1270699321 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc74006c600 0x7fc74006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7fc73b7fe640 1 -- 172.21.15.97:0/1270699321 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc74c08b7f0 con 0x7fc754166000 2024-03-29T04:07:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7fc752ffd640 1 --2- 172.21.15.97:0/1270699321 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc74006c600 0x7fc74006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7fc73b7fe640 1 --2- 172.21.15.97:0/1270699321 >> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] conn(0x7fc740074820 0x7fc740076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7fc73b7fe640 1 -- 172.21.15.97:0/1270699321 --> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7400772f0 con 0x7fc740074820 2024-03-29T04:07:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7fc73b7fe640 1 -- 172.21.15.97:0/1270699321 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc74c092050 con 0x7fc754166000 2024-03-29T04:07:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7f7aed7fa640 1 -- 172.21.15.97:0/2913553956 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7ae008a960 con 0x7f7af00cf200 2024-03-29T04:07:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7f7aed7fa640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] conn(0x7f7adc074820 0x7f7adc076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7f7aed7fa640 1 -- 172.21.15.97:0/2913553956 --> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ae0005ce0 con 0x7f7adc074820 2024-03-29T04:07:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7f7aed7fa640 1 -- 172.21.15.97:0/2913553956 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7ae008f2a0 con 0x7f7af00cf200 2024-03-29T04:07:45.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.925+0000 7f7aeffff640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7adc06c600 0x7f7adc06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7fdc8ffff640 1 --2- 172.21.15.97:0/3264736381 >> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] conn(0x7fdc7c074800 0x7fdc7c076c20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7f7af5637640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] conn(0x7f7adc074820 0x7f7adc076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7fdc8effd640 1 --2- 172.21.15.97:0/3264736381 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdc7c06c600 0x7fdc7c06eac0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdc84007fd0 tx=0x7fdc84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7fc752ffd640 1 --2- 172.21.15.97:0/1270699321 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc74006c600 0x7fc74006eac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc744036000 tx=0x7fc744030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7fc7527fc640 1 --2- 172.21.15.97:0/1270699321 >> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] conn(0x7fc740074820 0x7fc740076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7fb9f759e640 1 -- 172.21.15.97:0/2996283682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e0004800 msgr2=0x7fb9e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7fb9f759e640 1 --2- 172.21.15.97:0/2996283682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e0004800 0x7fb9e0004c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000e170 tx=0x7fb9e0005830 comp rx=0 tx=0).stop 2024-03-29T04:07:45.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7efc5affd640 1 -- 172.21.15.97:0/2255786642 <== osd.3 v2:172.21.15.97:6824/1504398439 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efc60022e50 con 0x7efc4c0747a0 2024-03-29T04:07:45.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.926+0000 7fdc77fff640 1 -- 172.21.15.97:0/3264736381 <== osd.5 v2:172.21.15.154:6808/2773817748 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc80034e70 con 0x7fdc7c074800 2024-03-29T04:07:45.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.927+0000 7fb9f6d9d640 1 -- 172.21.15.97:0/2996283682 reap_dead start 2024-03-29T04:07:45.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.927+0000 7fb9f759e640 1 -- 172.21.15.97:0/2996283682 shutdown_connections 2024-03-29T04:07:45.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.927+0000 7fb9f759e640 1 -- 172.21.15.97:0/2996283682 >> 172.21.15.97:0/2996283682 conn(0x7fb9f808bfc0 msgr2=0x7fb9f80d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.927+0000 7fb9f759e640 1 -- 172.21.15.97:0/2996283682 shutdown_connections 2024-03-29T04:07:45.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.927+0000 7fb9f759e640 1 -- 172.21.15.97:0/2996283682 wait complete. 2024-03-29T04:07:45.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.927+0000 7fc7527fc640 1 --2- 172.21.15.97:0/1270699321 >> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] conn(0x7fc740074820 0x7fc740076c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.928+0000 7fc73b7fe640 1 -- 172.21.15.97:0/1270699321 <== osd.0 v2:172.21.15.97:6800/2006100648 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc7400772f0 con 0x7fc740074820 2024-03-29T04:07:45.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.927+0000 7f7af5637640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] conn(0x7f7adc074820 0x7f7adc076c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.928+0000 7f7aeffff640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7adc06c600 0x7f7adc06eac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4000c00 tx=0x7f7ae4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.928+0000 7f7aed7fa640 1 -- 172.21.15.97:0/2913553956 <== osd.4 v2:172.21.15.154:6824/2981976178 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ae0005ce0 con 0x7f7adc074820 2024-03-29T04:07:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.929+0000 7fb9f759e640 1 Processor -- start 2024-03-29T04:07:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.929+0000 7fb9f759e640 1 -- start start 2024-03-29T04:07:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.930+0000 7fb9f759e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e0004800 0x7fb9f8161a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.930+0000 7fb9f759e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9f81f8b80 0x7fb9f815c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.930+0000 7fb9f759e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f809faf0 0x7fb9f809ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.930+0000 7fb9f759e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f80d27b0 con 0x7fb9f809faf0 2024-03-29T04:07:45.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.930+0000 7fb9f759e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f80d1670 con 0x7fb9e0004800 2024-03-29T04:07:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.930+0000 7fb9f759e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f80d33c0 con 0x7fb9f81f8b80 2024-03-29T04:07:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.931+0000 7fb9f659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e0004800 0x7fb9f8161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.931+0000 7fb9f659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e0004800 0x7fb9f8161a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55480/0 (socket says 172.21.15.97:55480) 2024-03-29T04:07:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.931+0000 7fb9f659c640 1 -- 172.21.15.97:0/3417184753 learned_addr learned my addr 172.21.15.97:0/3417184753 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.931+0000 7fb9f5d9b640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9f81f8b80 0x7fb9f815c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.931+0000 7fb9f6d9d640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f809faf0 0x7fb9f809ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.931+0000 7fb9f659c640 1 -- 172.21.15.97:0/3417184753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9f81f8b80 msgr2=0x7fb9f815c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.931+0000 7fb9f659c640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb9f81f8b80 0x7fb9f815c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.931+0000 7fb9f659c640 1 -- 172.21.15.97:0/3417184753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f809faf0 msgr2=0x7fb9f809ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.931+0000 7fb9f659c640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f809faf0 0x7fb9f809ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.931+0000 7fb9f659c640 1 -- 172.21.15.97:0/3417184753 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f81f8fc0 con 0x7fb9e0004800 2024-03-29T04:07:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.932+0000 7fb9f659c640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e0004800 0x7fb9f8161a50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800e730 tx=0x7fb9e800ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.932+0000 7fb9e6ffd640 1 -- 172.21.15.97:0/3417184753 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e800ee40 con 0x7fb9e0004800 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.932+0000 7fb9e6ffd640 1 -- 172.21.15.97:0/3417184753 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9e8018070 con 0x7fb9e0004800 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.933+0000 7fb9e6ffd640 1 -- 172.21.15.97:0/3417184753 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e800ee40 con 0x7fb9e0004800 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.933+0000 7fb9f759e640 1 -- 172.21.15.97:0/3417184753 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f809f6f0 con 0x7fb9e0004800 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.933+0000 7fb9f759e640 1 -- 172.21.15.97:0/3417184753 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f815ecd0 con 0x7fb9e0004800 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.936+0000 7fb9e6ffd640 1 -- 172.21.15.97:0/3417184753 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb9e800c8d0 con 0x7fb9e0004800 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.936+0000 7fb9e6ffd640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb9d806c930 0x7fb9d806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.937+0000 7fb9e6ffd640 1 -- 172.21.15.97:0/3417184753 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb9e8010040 con 0x7fb9e0004800 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.937+0000 7fb9e4ff9640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] conn(0x7fb9f815f7b0 0x7fb9f815fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.937+0000 7fb9f5d9b640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb9d806c930 0x7fb9d806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.937+0000 7fb9e4ff9640 1 -- 172.21.15.97:0/3417184753 --> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9f804f8f0 con 0x7fb9f815f7b0 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.938+0000 7fb9f6d9d640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] conn(0x7fb9f815f7b0 0x7fb9f815fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.938+0000 7fb9f5d9b640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb9d806c930 0x7fb9d806edf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0036000 tx=0x7fb9f002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.939+0000 7fb9f6d9d640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] conn(0x7fb9f815f7b0 0x7fb9f815fb90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:45.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.940+0000 7fb9e6ffd640 1 -- 172.21.15.97:0/3417184753 <== osd.6 v2:172.21.15.154:6800/1169845562 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9f804f8f0 con 0x7fb9f815f7b0 2024-03-29T04:07:45.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.953+0000 7f49bc115640 1 -- 172.21.15.97:0/1301994387 --> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f49b4068d50 con 0x7f4998074780 2024-03-29T04:07:45.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.954+0000 7f49aa7fc640 1 -- 172.21.15.97:0/1301994387 <== osd.2 v2:172.21.15.97:6816/127556154 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f49b4068d50 con 0x7f4998074780 2024-03-29T04:07:45.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.955+0000 7f4997fff640 1 -- 172.21.15.97:0/1301994387 >> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] conn(0x7f4998074780 msgr2=0x7f4998076ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.955+0000 7f4997fff640 1 --2- 172.21.15.97:0/1301994387 >> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] conn(0x7f4998074780 0x7f4998076ba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.955+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 --> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcb34002d50 con 0x7fcb38074850 2024-03-29T04:07:45.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.955+0000 7f4997fff640 1 -- 172.21.15.97:0/1301994387 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f499806c580 msgr2=0x7f499806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.955+0000 7f4997fff640 1 --2- 172.21.15.97:0/1301994387 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f499806c580 0x7f499806ea40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009c00 tx=0x7f49b0009190 comp rx=0 tx=0).stop 2024-03-29T04:07:45.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.955+0000 7f4997fff640 1 -- 172.21.15.97:0/1301994387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49b40d2c80 msgr2=0x7f49b40cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.955+0000 7f4997fff640 1 --2- 172.21.15.97:0/1301994387 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49b40d2c80 0x7f49b40cd940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f49a4033000 tx=0x7f49a40046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.955+0000 7f49ba6b2640 1 -- 172.21.15.97:0/1301994387 reap_dead start 2024-03-29T04:07:45.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.955+0000 7f4997fff640 1 -- 172.21.15.97:0/1301994387 shutdown_connections 2024-03-29T04:07:45.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.955+0000 7f4997fff640 1 -- 172.21.15.97:0/1301994387 >> 172.21.15.97:0/1301994387 conn(0x7f49b408bfc0 msgr2=0x7f49b40b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.955+0000 7f4997fff640 1 -- 172.21.15.97:0/1301994387 shutdown_connections 2024-03-29T04:07:45.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.955+0000 7f4997fff640 1 -- 172.21.15.97:0/1301994387 wait complete. 2024-03-29T04:07:45.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.956+0000 7fcb49ffb640 1 -- 172.21.15.97:0/3244344505 <== osd.7 v2:172.21.15.154:6816/2375296016 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcb34002d50 con 0x7fcb38074850 2024-03-29T04:07:45.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.956+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] conn(0x7fcb38074850 msgr2=0x7fcb38076c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.956+0000 7fcb5b51b640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] conn(0x7fcb38074850 0x7fcb38076c70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.957+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcb3806c630 msgr2=0x7fcb3806eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.957+0000 7fcb5b51b640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcb3806c630 0x7fcb3806eaf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000c00 tx=0x7fcb50009030 comp rx=0 tx=0).stop 2024-03-29T04:07:45.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.957+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb541617b0 msgr2=0x7fcb540b9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.957+0000 7fcb5b51b640 1 --2- 172.21.15.97:0/3244344505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb541617b0 0x7fcb540b9360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcb44010790 tx=0x7fcb44010c50 comp rx=0 tx=0).stop 2024-03-29T04:07:45.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.957+0000 7fcb59ab8640 1 -- 172.21.15.97:0/3244344505 reap_dead start 2024-03-29T04:07:45.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.957+0000 7f79be70e640 1 -- 172.21.15.97:0/1982087663 --> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f799c002dd0 con 0x7f79980747a0 2024-03-29T04:07:45.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.957+0000 7f79b4ff9640 1 -- 172.21.15.97:0/1982087663 <== osd.1 v2:172.21.15.97:6808/4164988844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f799c002dd0 con 0x7f79980747a0 2024-03-29T04:07:45.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.957+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 shutdown_connections 2024-03-29T04:07:45.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.957+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 >> 172.21.15.97:0/3244344505 conn(0x7fcb5408bfc0 msgr2=0x7fcb540b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.957+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 shutdown_connections 2024-03-29T04:07:45.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.957+0000 7fcb5b51b640 1 -- 172.21.15.97:0/3244344505 wait complete. 2024-03-29T04:07:45.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.958+0000 7f79ae7fc640 1 -- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] conn(0x7f79980747a0 msgr2=0x7f7998076bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.958+0000 7f79ae7fc640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] conn(0x7f79980747a0 0x7f7998076bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.958+0000 7f79ae7fc640 1 -- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f799806c600 msgr2=0x7f799806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.958+0000 7f79ae7fc640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f799806c600 0x7f799806eac0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f79a8036000 tx=0x7f79a8030000 comp rx=0 tx=0).stop 2024-03-29T04:07:45.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.958+0000 7f79ae7fc640 1 -- 172.21.15.97:0/1982087663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b815d140 msgr2=0x7f79b80b0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.958+0000 7f79ae7fc640 1 --2- 172.21.15.97:0/1982087663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b815d140 0x7f79b80b0c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f79b000c3c0 tx=0x7f79b000c880 comp rx=0 tx=0).stop 2024-03-29T04:07:45.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.958+0000 7f79bccab640 1 -- 172.21.15.97:0/1982087663 reap_dead start 2024-03-29T04:07:45.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.958+0000 7f79ae7fc640 1 -- 172.21.15.97:0/1982087663 shutdown_connections 2024-03-29T04:07:45.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.958+0000 7f79ae7fc640 1 -- 172.21.15.97:0/1982087663 >> 172.21.15.97:0/1982087663 conn(0x7f79b806b6f0 msgr2=0x7f79b815b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:45.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.959+0000 7f79ae7fc640 1 -- 172.21.15.97:0/1982087663 shutdown_connections 2024-03-29T04:07:45.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.959+0000 7f79ae7fc640 1 -- 172.21.15.97:0/1982087663 wait complete. 2024-03-29T04:07:45.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.959+0000 7fc753fff640 1 -- 172.21.15.97:0/1270699321 --> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc734002dd0 con 0x7fc740074820 2024-03-29T04:07:45.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.960+0000 7efc70b15640 1 -- 172.21.15.97:0/2255786642 --> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efc640e00f0 con 0x7efc4c0747a0 2024-03-29T04:07:45.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.960+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 --> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdc70002dd0 con 0x7fdc7c074800 2024-03-29T04:07:45.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.960+0000 7fc73b7fe640 1 -- 172.21.15.97:0/1270699321 <== osd.0 v2:172.21.15.97:6800/2006100648 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc734002dd0 con 0x7fc740074820 2024-03-29T04:07:45.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.960+0000 7fdc77fff640 1 -- 172.21.15.97:0/3264736381 <== osd.5 v2:172.21.15.154:6808/2773817748 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdc70002dd0 con 0x7fdc7c074800 2024-03-29T04:07:45.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.960+0000 7efc5affd640 1 -- 172.21.15.97:0/2255786642 <== osd.3 v2:172.21.15.97:6824/1504398439 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efc640e00f0 con 0x7efc4c0747a0 2024-03-29T04:07:45.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.960+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 >> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] conn(0x7fdc7c074800 msgr2=0x7fdc7c076c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.960+0000 7fdc95bcd640 1 --2- 172.21.15.97:0/3264736381 >> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] conn(0x7fdc7c074800 0x7fdc7c076c20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7efc70b15640 1 -- 172.21.15.97:0/2255786642 >> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] conn(0x7efc4c0747a0 msgr2=0x7efc4c076bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7efc70b15640 1 --2- 172.21.15.97:0/2255786642 >> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] conn(0x7efc4c0747a0 0x7efc4c076bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7efc70b15640 1 -- 172.21.15.97:0/2255786642 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7efc4c06c600 msgr2=0x7efc4c06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7efc70b15640 1 --2- 172.21.15.97:0/2255786642 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7efc4c06c600 0x7efc4c06eac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc5c006fd0 tx=0x7efc5c009060 comp rx=0 tx=0).stop 2024-03-29T04:07:45.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7efc70b15640 1 -- 172.21.15.97:0/2255786642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc64048c30 msgr2=0x7efc640d8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7efc70b15640 1 --2- 172.21.15.97:0/2255786642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc64048c30 0x7efc640d8e20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efc6000e7e0 tx=0x7efc6000eca0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fc753fff640 1 -- 172.21.15.97:0/1270699321 >> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] conn(0x7fc740074820 msgr2=0x7fc740076c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fc753fff640 1 --2- 172.21.15.97:0/1270699321 >> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] conn(0x7fc740074820 0x7fc740076c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fc753fff640 1 -- 172.21.15.97:0/1270699321 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc74006c600 msgr2=0x7fc74006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fc753fff640 1 --2- 172.21.15.97:0/1270699321 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc74006c600 0x7fc74006eac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc744036000 tx=0x7fc744030000 comp rx=0 tx=0).stop 2024-03-29T04:07:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fc753fff640 1 -- 172.21.15.97:0/1270699321 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc754166000 msgr2=0x7fc75409a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fc753fff640 1 --2- 172.21.15.97:0/1270699321 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc754166000 0x7fc75409a2b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00c780 tx=0x7fc74c00cc40 comp rx=0 tx=0).stop 2024-03-29T04:07:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdc7c06c600 msgr2=0x7fdc7c06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fc7537fe640 1 -- 172.21.15.97:0/1270699321 reap_dead start 2024-03-29T04:07:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fdc95bcd640 1 --2- 172.21.15.97:0/3264736381 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdc7c06c600 0x7fdc7c06eac0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdc84007fd0 tx=0x7fdc84009040 comp rx=0 tx=0).stop 2024-03-29T04:07:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fc753fff640 1 -- 172.21.15.97:0/1270699321 shutdown_connections 2024-03-29T04:07:46.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fc753fff640 1 -- 172.21.15.97:0/1270699321 >> 172.21.15.97:0/1270699321 conn(0x7fc7540b22e0 msgr2=0x7fc7540b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc9006cc60 msgr2=0x7fdc900cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fdc95bcd640 1 --2- 172.21.15.97:0/3264736381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc9006cc60 0x7fdc900cd960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdc800098a0 tx=0x7fdc800046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fdc8ffff640 1 -- 172.21.15.97:0/3264736381 reap_dead start 2024-03-29T04:07:46.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fc753fff640 1 -- 172.21.15.97:0/1270699321 shutdown_connections 2024-03-29T04:07:46.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7efc6ad9d640 1 -- 172.21.15.97:0/2255786642 reap_dead start 2024-03-29T04:07:46.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7fc753fff640 1 -- 172.21.15.97:0/1270699321 wait complete. 2024-03-29T04:07:46.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7efc70b15640 1 -- 172.21.15.97:0/2255786642 shutdown_connections 2024-03-29T04:07:46.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.961+0000 7efc70b15640 1 -- 172.21.15.97:0/2255786642 >> 172.21.15.97:0/2255786642 conn(0x7efc6401ef30 msgr2=0x7efc6403b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.962+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 shutdown_connections 2024-03-29T04:07:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.962+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 >> 172.21.15.97:0/3264736381 conn(0x7fdc9008bfc0 msgr2=0x7fdc9015dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.962+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 shutdown_connections 2024-03-29T04:07:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.962+0000 7fdc95bcd640 1 -- 172.21.15.97:0/3264736381 wait complete. 2024-03-29T04:07:46.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.962+0000 7efc70b15640 1 -- 172.21.15.97:0/2255786642 shutdown_connections 2024-03-29T04:07:46.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.962+0000 7efc70b15640 1 -- 172.21.15.97:0/2255786642 wait complete. 2024-03-29T04:07:46.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.964+0000 7f7af709a640 1 -- 172.21.15.97:0/2913553956 --> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7af0162a20 con 0x7f7adc074820 2024-03-29T04:07:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.964+0000 7f7aed7fa640 1 -- 172.21.15.97:0/2913553956 <== osd.4 v2:172.21.15.154:6824/2981976178 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7af0162a20 con 0x7f7adc074820 2024-03-29T04:07:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.965+0000 7f7af709a640 1 -- 172.21.15.97:0/2913553956 >> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] conn(0x7f7adc074820 msgr2=0x7f7adc076c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.965+0000 7f7af709a640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] conn(0x7f7adc074820 0x7f7adc076c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.966+0000 7f7af709a640 1 -- 172.21.15.97:0/2913553956 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7adc06c600 msgr2=0x7f7adc06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.966+0000 7f7af709a640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7adc06c600 0x7f7adc06eac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4000c00 tx=0x7f7ae4008030 comp rx=0 tx=0).stop 2024-03-29T04:07:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.966+0000 7f7af709a640 1 -- 172.21.15.97:0/2913553956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af00cf200 msgr2=0x7f7af01f7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.966+0000 7f7af709a640 1 --2- 172.21.15.97:0/2913553956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7af00cf200 0x7f7af01f7b10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000c430 tx=0x7f7ae000c8f0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.966+0000 7f7af5637640 1 -- 172.21.15.97:0/2913553956 reap_dead start 2024-03-29T04:07:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.966+0000 7f7af709a640 1 -- 172.21.15.97:0/2913553956 shutdown_connections 2024-03-29T04:07:46.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.966+0000 7f7af709a640 1 -- 172.21.15.97:0/2913553956 >> 172.21.15.97:0/2913553956 conn(0x7f7af00b56e0 msgr2=0x7f7af0163d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.968+0000 7f7af709a640 1 -- 172.21.15.97:0/2913553956 shutdown_connections 2024-03-29T04:07:46.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.969+0000 7f7af709a640 1 -- 172.21.15.97:0/2913553956 wait complete. 2024-03-29T04:07:46.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.969+0000 7fb9e4ff9640 1 -- 172.21.15.97:0/3417184753 --> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb9f80d51c0 con 0x7fb9f815f7b0 2024-03-29T04:07:46.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.969+0000 7fb9e6ffd640 1 -- 172.21.15.97:0/3417184753 <== osd.6 v2:172.21.15.154:6800/1169845562 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb9f80d51c0 con 0x7fb9f815f7b0 2024-03-29T04:07:46.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.970+0000 7fb9f759e640 1 -- 172.21.15.97:0/3417184753 >> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] conn(0x7fb9f815f7b0 msgr2=0x7fb9f815fb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.970+0000 7fb9f759e640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] conn(0x7fb9f815f7b0 0x7fb9f815fb90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.970+0000 7fb9f759e640 1 -- 172.21.15.97:0/3417184753 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb9d806c930 msgr2=0x7fb9d806edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.970+0000 7fb9f759e640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb9d806c930 0x7fb9d806edf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0036000 tx=0x7fb9f002e000 comp rx=0 tx=0).stop 2024-03-29T04:07:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.971+0000 7fb9f759e640 1 -- 172.21.15.97:0/3417184753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e0004800 msgr2=0x7fb9f8161a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.971+0000 7fb9f759e640 1 --2- 172.21.15.97:0/3417184753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e0004800 0x7fb9f8161a50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800e730 tx=0x7fb9e800ebf0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.971+0000 7fb9f6d9d640 1 -- 172.21.15.97:0/3417184753 reap_dead start 2024-03-29T04:07:46.009 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2024-03-29T04:07:46.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-29T04:07:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.976+0000 7fb9f759e640 1 -- 172.21.15.97:0/3417184753 shutdown_connections 2024-03-29T04:07:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.976+0000 7fb9f759e640 1 -- 172.21.15.97:0/3417184753 >> 172.21.15.97:0/3417184753 conn(0x7fb9f808bfc0 msgr2=0x7fb9f80ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.977+0000 7fb9f759e640 1 -- 172.21.15.97:0/3417184753 shutdown_connections 2024-03-29T04:07:46.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:45.977+0000 7fb9f759e640 1 -- 172.21.15.97:0/3417184753 wait complete. 2024-03-29T04:07:46.010 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2024-03-29T04:07:46.011 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2024-03-29T04:07:46.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-29T04:07:46.012 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-29T04:07:46.012 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2024-03-29T04:07:46.013 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2024-03-29T04:07:46.013 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-29T04:07:46.014 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-29T04:07:46.014 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2024-03-29T04:07:46.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-29T04:07:46.015 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2024-03-29T04:07:46.016 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-29T04:07:46.017 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2024-03-29T04:07:46.017 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-29T04:07:46.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.241+0000 7f47d02d9640 1 Processor -- start 2024-03-29T04:07:46.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.241+0000 7f47d02d9640 1 -- start start 2024-03-29T04:07:46.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.242+0000 7fab58d50640 1 Processor -- start 2024-03-29T04:07:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.242+0000 7fc2aa960640 1 Processor -- start 2024-03-29T04:07:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.242+0000 7fc2aa960640 1 -- start start 2024-03-29T04:07:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.242+0000 7f47d02d9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47c80b4ed0 0x7f47c80cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.242+0000 7f47d02d9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f47c815c670 0x7f47c815e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.242+0000 7f47d02d9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c815d650 con 0x7f47c80b4710 2024-03-29T04:07:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.242+0000 7f47d02d9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f47c80b64d0 con 0x7f47c80b4ed0 2024-03-29T04:07:46.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.242+0000 7f47d02d9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f47c815df80 con 0x7f47c815c670 2024-03-29T04:07:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7f7e86adb640 1 Processor -- start 2024-03-29T04:07:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7f89909ba640 1 Processor -- start 2024-03-29T04:07:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.242+0000 7fab58d50640 1 -- start start 2024-03-29T04:07:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fab58d50640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fab540b30b0 0x7fab540c5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fab51d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fab540cd280 0x7fab540c5020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50428/0 (socket says 172.21.15.97:50428) 2024-03-29T04:07:46.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fab58d50640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab541609c0 con 0x7fab540cd280 2024-03-29T04:07:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fab58d50640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab540ad130 con 0x7fab541605e0 2024-03-29T04:07:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fab58d50640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fab540add40 con 0x7fab540b30b0 2024-03-29T04:07:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fab51d9b640 1 -- 172.21.15.97:0/2327693775 learned_addr learned my addr 172.21.15.97:0/2327693775 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fab52d9d640 1 --2- 172.21.15.97:0/2327693775 >> v2:172.21.15.154:3300/0 conn(0x7fab540b30b0 0x7fab540c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fab5159a640 1 -- 172.21.15.97:0/2327693775 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 622113541 0 0) 0x7fab540ad130 con 0x7fab541605e0 2024-03-29T04:07:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fab5159a640 1 -- 172.21.15.97:0/2327693775 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab3c003520 con 0x7fab541605e0 2024-03-29T04:07:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7fc2aa960640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2a40cd350 0x7fc2a4160060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7fc2aa960640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc2a4099c50 0x7fc2a406d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7fc2aa960640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2a40c5490 con 0x7fc2a406dd90 2024-03-29T04:07:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7fc2aa960640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fc2a40c5770 con 0x7fc2a4099c50 2024-03-29T04:07:46.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7fc2aa960640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a40c5a50 con 0x7fc2a40cd350 2024-03-29T04:07:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7fc2a37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc2a4099c50 0x7fc2a406d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7fc2a37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc2a4099c50 0x7fc2a406d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38914/0 (socket says 172.21.15.97:38914) 2024-03-29T04:07:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7fc2a37fe640 1 -- 172.21.15.97:0/3525382961 learned_addr learned my addr 172.21.15.97:0/3525382961 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fc2a37fe640 1 -- 172.21.15.97:0/3525382961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2a40cd350 msgr2=0x7fc2a4160060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:07:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fc2a37fe640 1 --2- 172.21.15.97:0/3525382961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2a40cd350 0x7fc2a4160060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fc2a37fe640 1 -- 172.21.15.97:0/3525382961 >> v1:172.21.15.154:6789/0 conn(0x7fc2a406dd90 legacy=0x7fc2a40c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fc2a37fe640 1 -- 172.21.15.97:0/3525382961 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a40c5d30 con 0x7fc2a4099c50 2024-03-29T04:07:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7f7e86adb640 1 -- start start 2024-03-29T04:07:46.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f7e86adb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7e800b4a50 0x7f7e80166f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7f47ce075640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47c80b4ed0 0x7f47c80cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7f47cd874640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f47c80b4710 0x7f47c80b52b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50420/0 (socket says 172.21.15.97:50420) 2024-03-29T04:07:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7f47cd874640 1 -- 172.21.15.97:0/1509801676 learned_addr learned my addr 172.21.15.97:0/1509801676 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7f47ce075640 1 -- 172.21.15.97:0/1509801676 >> v2:172.21.15.154:3300/0 conn(0x7f47c815c670 msgr2=0x7f47c815e410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:07:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7f47ce075640 1 --2- 172.21.15.97:0/1509801676 >> v2:172.21.15.154:3300/0 conn(0x7f47c815c670 0x7f47c815e410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7f47ce075640 1 -- 172.21.15.97:0/1509801676 >> v1:172.21.15.97:6789/0 conn(0x7f47c80b4710 legacy=0x7f47c80b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7f47ce075640 1 -- 172.21.15.97:0/1509801676 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c815d930 con 0x7f47c80b4ed0 2024-03-29T04:07:46.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.243+0000 7f89909ba640 1 -- start start 2024-03-29T04:07:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f89909ba640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f898c0b5e60 0x7f898c0b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f89909ba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f898c0b6240 0x7f898c0d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f89909ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f898c0b7b30 con 0x7f898c0cf440 2024-03-29T04:07:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f89909ba640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f898c0d28a0 con 0x7f898c0b5e60 2024-03-29T04:07:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f89909ba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f898c0cfa00 con 0x7f898c0b6240 2024-03-29T04:07:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fb2a3c7b640 1 Processor -- start 2024-03-29T04:07:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f7e86adb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e80167470 0x7f7e800cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f7e86adb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e800cbb30 con 0x7f7e80162d10 2024-03-29T04:07:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f7e86adb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f7e80164100 con 0x7f7e800b4a50 2024-03-29T04:07:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f7e86adb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7e800b3b60 con 0x7f7e80167470 2024-03-29T04:07:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f7e85ad9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7e80162d10 0x7f7e80166820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50444/0 (socket says 172.21.15.97:50444) 2024-03-29T04:07:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7f7e85ad9640 1 -- 172.21.15.97:0/1622251353 learned_addr learned my addr 172.21.15.97:0/1622251353 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7fdcc614d640 1 Processor -- start 2024-03-29T04:07:46.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f898a59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f898c0b5e60 0x7f898c0b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7f7e862da640 1 --2- 172.21.15.97:0/1622251353 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e80167470 0x7f7e800cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f898a59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f898c0b5e60 0x7f898c0b75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38922/0 (socket says 172.21.15.97:38922) 2024-03-29T04:07:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.244+0000 7f898a59c640 1 -- 172.21.15.97:0/3148990665 learned_addr learned my addr 172.21.15.97:0/3148990665 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:46.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7f898ad9d640 1 --2- 172.21.15.97:0/3148990665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f898c0b6240 0x7f898c0d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7f7e862da640 1 -- 172.21.15.97:0/1622251353 >> v2:172.21.15.97:3300/0 conn(0x7f7e800b4a50 msgr2=0x7f7e80166f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7f7e862da640 1 --2- 172.21.15.97:0/1622251353 >> v2:172.21.15.97:3300/0 conn(0x7f7e800b4a50 0x7f7e80166f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7f7e862da640 1 -- 172.21.15.97:0/1622251353 >> v1:172.21.15.97:6789/0 conn(0x7f7e80162d10 legacy=0x7f7e80166820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7f7e862da640 1 -- 172.21.15.97:0/1622251353 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e800b3e40 con 0x7f7e80167470 2024-03-29T04:07:46.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7fb2a3c7b640 1 -- start start 2024-03-29T04:07:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7fdcc614d640 1 -- start start 2024-03-29T04:07:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7f898a59c640 1 -- 172.21.15.97:0/3148990665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f898c0b6240 msgr2=0x7f898c0d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7f898a59c640 1 --2- 172.21.15.97:0/3148990665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f898c0b6240 0x7f898c0d35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7f898a59c640 1 -- 172.21.15.97:0/3148990665 >> v1:172.21.15.154:6789/0 conn(0x7f898c0cf440 legacy=0x7f898c06d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7f898a59c640 1 -- 172.21.15.97:0/3148990665 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c0d2ad0 con 0x7f898c0b5e60 2024-03-29T04:07:46.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7f898ad9d640 1 --2- 172.21.15.97:0/3148990665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f898c0b6240 0x7f898c0d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:46.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7fab5159a640 1 -- 172.21.15.97:0/2327693775 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3780734528 0 0) 0x7fab541609c0 con 0x7fab540cd280 2024-03-29T04:07:46.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7fab52d9d640 1 -- 172.21.15.97:0/2327693775 >> v1:172.21.15.154:6789/0 conn(0x7fab541605e0 legacy=0x7fab540ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7fab52d9d640 1 -- 172.21.15.97:0/2327693775 >> v1:172.21.15.97:6789/0 conn(0x7fab540cd280 legacy=0x7fab540c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7fab52d9d640 1 -- 172.21.15.97:0/2327693775 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab540ad410 con 0x7fab540b30b0 2024-03-29T04:07:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.245+0000 7fc2a37fe640 1 --2- 172.21.15.97:0/3525382961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a4099c50 0x7fc2a406d850 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc294009870 tx=0x7fc29402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988cf456b9c98e2c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7fc2a2ffd640 1 -- 172.21.15.97:0/3525382961 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294035020 con 0x7fc2a4099c50 2024-03-29T04:07:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7fc2a2ffd640 1 -- 172.21.15.97:0/3525382961 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc294031870 con 0x7fc2a4099c50 2024-03-29T04:07:46.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7fc2a2ffd640 1 -- 172.21.15.97:0/3525382961 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294035020 con 0x7fc2a4099c50 2024-03-29T04:07:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7fb2a3c7b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb29c0cd280 0x7fb29c0b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7fb2a3c7b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb29c0c5020 0x7fb29c0c5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7fb2a3c7b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb29c1609c0 con 0x7fb29c1605e0 2024-03-29T04:07:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7fb2a3c7b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fb29c0add40 con 0x7fb29c0c5020 2024-03-29T04:07:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7fb2a3c7b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fb29c0ad130 con 0x7fb29c0cd280 2024-03-29T04:07:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7fb2a1a17640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb29c1605e0 0x7fb29c0ccb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:40442/0 (socket says 172.21.15.97:40442) 2024-03-29T04:07:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7fb2a1a17640 1 -- 172.21.15.97:0/25901902 learned_addr learned my addr 172.21.15.97:0/25901902 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7fb2a1216640 1 --2- 172.21.15.97:0/25901902 >> v2:172.21.15.154:3300/0 conn(0x7fb29c0cd280 0x7fb29c0b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7f7e862da640 1 --2- 172.21.15.97:0/1622251353 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e80167470 0x7f7e800cc010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009d90 tx=0x7f7e7802bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fb2648f1307f927 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fb2a0a15640 1 -- 172.21.15.97:0/25901902 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 189897789 0 0) 0x7fb29c1609c0 con 0x7fb29c1605e0 2024-03-29T04:07:46.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7f47ce075640 1 --2- 172.21.15.97:0/1509801676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47c80b4ed0 0x7f47c80cd9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f47c80727c0 tx=0x7f47c0034450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86dca739e55d5aa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7f47cd073640 1 -- 172.21.15.97:0/1509801676 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0039710 con 0x7f47c80b4ed0 2024-03-29T04:07:46.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fb2a0a15640 1 -- 172.21.15.97:0/25901902 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb288003520 con 0x7fb29c1605e0 2024-03-29T04:07:46.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7f47cd073640 1 -- 172.21.15.97:0/1509801676 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c00398b0 con 0x7f47c80b4ed0 2024-03-29T04:07:46.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7f47cd073640 1 -- 172.21.15.97:0/1509801676 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0039c80 con 0x7f47c80b4ed0 2024-03-29T04:07:46.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fb2a1216640 1 -- 172.21.15.97:0/25901902 >> v2:172.21.15.97:3300/0 conn(0x7fb29c0c5020 msgr2=0x7fb29c0c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:46.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.246+0000 7f898a59c640 1 --2- 172.21.15.97:0/3148990665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c0b5e60 0x7f898c0b75f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8984009e50 tx=0x7f898402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93829af2790d4473 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fb2a2218640 1 --2- 172.21.15.97:0/25901902 >> v2:172.21.15.97:3300/0 conn(0x7fb29c0c5020 0x7fb29c0c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fab52d9d640 1 --2- 172.21.15.97:0/2327693775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab540b30b0 0x7fab540c5730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009d90 tx=0x7fab4c02bc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b39e9dce16c0b3d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7f898959a640 1 -- 172.21.15.97:0/3148990665 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898402c9f0 con 0x7f898c0b5e60 2024-03-29T04:07:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7f898959a640 1 -- 172.21.15.97:0/3148990665 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f898402cb90 con 0x7f898c0b5e60 2024-03-29T04:07:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7f898959a640 1 -- 172.21.15.97:0/3148990665 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89840343f0 con 0x7f898c0b5e60 2024-03-29T04:07:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fdcc614d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdcc0166ea0 0x7fdcc0162f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fdcc614d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc00acf20 0x7fdcc00ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fdcc614d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc0163ee0 con 0x7fdcc00cca20 2024-03-29T04:07:46.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fdcc614d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc00ce680 con 0x7fdcc0166ea0 2024-03-29T04:07:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fdcc614d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc0163920 con 0x7fdcc00acf20 2024-03-29T04:07:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fdcc514b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdcc00cca20 0x7fdcc00cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50450/0 (socket says 172.21.15.97:50450) 2024-03-29T04:07:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fc2aa960640 1 -- 172.21.15.97:0/3525382961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a4099c50 msgr2=0x7fc2a406d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fc2aa960640 1 --2- 172.21.15.97:0/3525382961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a4099c50 0x7fc2a406d850 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc294009870 tx=0x7fc29402b910 comp rx=0 tx=0).stop 2024-03-29T04:07:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fc2aa960640 1 -- 172.21.15.97:0/3525382961 shutdown_connections 2024-03-29T04:07:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fc2aa960640 1 --2- 172.21.15.97:0/3525382961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a4099c50 0x7fc2a406d850 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fc2aa960640 1 --2- 172.21.15.97:0/3525382961 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2a40cd350 0x7fc2a4160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fc2aa960640 1 -- 172.21.15.97:0/3525382961 >> 172.21.15.97:0/3525382961 conn(0x7fc2a4072ba0 msgr2=0x7fc2a4072f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fdcc514b640 1 -- 172.21.15.97:0/1563909601 learned_addr learned my addr 172.21.15.97:0/1563909601 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:46.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fdcc494a640 1 --2- 172.21.15.97:0/1563909601 >> v2:172.21.15.97:3300/0 conn(0x7fdcc0166ea0 0x7fdcc0162f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fdcc594c640 1 --2- 172.21.15.97:0/1563909601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc00acf20 0x7fdcc00ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fab5159a640 1 -- 172.21.15.97:0/2327693775 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c039070 con 0x7fab540b30b0 2024-03-29T04:07:46.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fab5159a640 1 -- 172.21.15.97:0/2327693775 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab4c02c790 con 0x7fab540b30b0 2024-03-29T04:07:46.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fab5159a640 1 -- 172.21.15.97:0/2327693775 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab44001520 con 0x7fab540b30b0 2024-03-29T04:07:46.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f47d02d9640 1 -- 172.21.15.97:0/1509801676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47c80b4ed0 msgr2=0x7f47c80cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f47d02d9640 1 --2- 172.21.15.97:0/1509801676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47c80b4ed0 0x7f47c80cd9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f47c80727c0 tx=0x7f47c0034450 comp rx=0 tx=0).stop 2024-03-29T04:07:46.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fdcc594c640 1 -- 172.21.15.97:0/1563909601 >> v2:172.21.15.97:3300/0 conn(0x7fdcc0166ea0 msgr2=0x7fdcc0162f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:46.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fdcc594c640 1 --2- 172.21.15.97:0/1563909601 >> v2:172.21.15.97:3300/0 conn(0x7fdcc0166ea0 0x7fdcc0162f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fdcc594c640 1 -- 172.21.15.97:0/1563909601 >> v1:172.21.15.97:6789/0 conn(0x7fdcc00cca20 legacy=0x7fdcc00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7fdcc594c640 1 -- 172.21.15.97:0/1563909601 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc0163c00 con 0x7fdcc00acf20 2024-03-29T04:07:46.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f89909ba640 1 -- 172.21.15.97:0/3148990665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c0b5e60 msgr2=0x7f898c0b75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f89909ba640 1 --2- 172.21.15.97:0/3148990665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c0b5e60 0x7f898c0b75f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8984009e50 tx=0x7f898402be50 comp rx=0 tx=0).stop 2024-03-29T04:07:46.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fb2a1216640 1 --2- 172.21.15.97:0/25901902 >> v2:172.21.15.97:3300/0 conn(0x7fb29c0c5020 0x7fb29c0c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fb2a1216640 1 -- 172.21.15.97:0/25901902 >> v1:172.21.15.154:6789/0 conn(0x7fb29c1605e0 legacy=0x7fb29c0ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7fb2a1216640 1 -- 172.21.15.97:0/25901902 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c0ad410 con 0x7fb29c0cd280 2024-03-29T04:07:46.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7fb2a2218640 1 --2- 172.21.15.97:0/25901902 >> v2:172.21.15.97:3300/0 conn(0x7fb29c0c5020 0x7fb29c0c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:07:46.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7fc2aa960640 1 -- 172.21.15.97:0/3525382961 shutdown_connections 2024-03-29T04:07:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7fc2aa960640 1 -- 172.21.15.97:0/3525382961 wait complete. 2024-03-29T04:07:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7f7e84ad7640 1 -- 172.21.15.97:0/1622251353 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78039070 con 0x7f7e80167470 2024-03-29T04:07:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.247+0000 7f7e84ad7640 1 -- 172.21.15.97:0/1622251353 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e7802cb40 con 0x7f7e80167470 2024-03-29T04:07:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f7e86adb640 1 -- 172.21.15.97:0/1622251353 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e80167470 msgr2=0x7f7e800cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f7e86adb640 1 --2- 172.21.15.97:0/1622251353 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e80167470 0x7f7e800cc010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009d90 tx=0x7f7e7802bd90 comp rx=0 tx=0).stop 2024-03-29T04:07:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f7e86adb640 1 -- 172.21.15.97:0/1622251353 shutdown_connections 2024-03-29T04:07:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f7e86adb640 1 --2- 172.21.15.97:0/1622251353 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e80167470 0x7f7e800cc010 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f7e86adb640 1 --2- 172.21.15.97:0/1622251353 >> v2:172.21.15.97:3300/0 conn(0x7f7e800b4a50 0x7f7e80166f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f7e86adb640 1 -- 172.21.15.97:0/1622251353 >> 172.21.15.97:0/1622251353 conn(0x7f7e800b1cb0 msgr2=0x7f7e800b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f7e86adb640 1 -- 172.21.15.97:0/1622251353 shutdown_connections 2024-03-29T04:07:46.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f7e86adb640 1 -- 172.21.15.97:0/1622251353 wait complete. 2024-03-29T04:07:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f47d02d9640 1 -- 172.21.15.97:0/1509801676 shutdown_connections 2024-03-29T04:07:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f47d02d9640 1 --2- 172.21.15.97:0/1509801676 >> v2:172.21.15.154:3300/0 conn(0x7f47c815c670 0x7f47c815e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f47d02d9640 1 --2- 172.21.15.97:0/1509801676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47c80b4ed0 0x7f47c80cd9a0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f47c80727c0 tx=0x7f47c0034450 comp rx=0 tx=0).stop 2024-03-29T04:07:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f47d02d9640 1 -- 172.21.15.97:0/1509801676 >> 172.21.15.97:0/1509801676 conn(0x7f47c8072d70 msgr2=0x7f47c80b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f47d02d9640 1 -- 172.21.15.97:0/1509801676 shutdown_connections 2024-03-29T04:07:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7fab58d50640 1 -- 172.21.15.97:0/2327693775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab540b30b0 msgr2=0x7fab540c5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7fab58d50640 1 --2- 172.21.15.97:0/2327693775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab540b30b0 0x7fab540c5730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009d90 tx=0x7fab4c02bc60 comp rx=0 tx=0).stop 2024-03-29T04:07:46.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f89909ba640 1 -- 172.21.15.97:0/3148990665 shutdown_connections 2024-03-29T04:07:46.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f89909ba640 1 --2- 172.21.15.97:0/3148990665 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f898c0b6240 0x7f898c0d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f89909ba640 1 --2- 172.21.15.97:0/3148990665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c0b5e60 0x7f898c0b75f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f89909ba640 1 -- 172.21.15.97:0/3148990665 >> 172.21.15.97:0/3148990665 conn(0x7f898c0d0e60 msgr2=0x7f898c0d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7f89909ba640 1 -- 172.21.15.97:0/3148990665 shutdown_connections 2024-03-29T04:07:46.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7fb2a1216640 1 --2- 172.21.15.97:0/25901902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29c0cd280 0x7fb29c0b30b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb29c099500 tx=0x7fb28c02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96e7dac0443194c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7fab58d50640 1 -- 172.21.15.97:0/2327693775 shutdown_connections 2024-03-29T04:07:46.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7fab58d50640 1 --2- 172.21.15.97:0/2327693775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab540b30b0 0x7fab540c5730 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.248+0000 7fab58d50640 1 -- 172.21.15.97:0/2327693775 >> 172.21.15.97:0/2327693775 conn(0x7fab5406b510 msgr2=0x7fab5406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.249+0000 7fdcc594c640 1 --2- 172.21.15.97:0/1563909601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc00acf20 0x7fdcc00ad3b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0097a0 tx=0x7fdcbc02b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4dc5a63c9f6b7bd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.249+0000 7f47d02d9640 1 -- 172.21.15.97:0/1509801676 wait complete. 2024-03-29T04:07:46.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.249+0000 7f920ec70640 1 Processor -- start 2024-03-29T04:07:46.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.249+0000 7fb2a0a15640 1 -- 172.21.15.97:0/25901902 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0317c0 con 0x7fb29c0cd280 2024-03-29T04:07:46.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.249+0000 7f920ec70640 1 -- start start 2024-03-29T04:07:46.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.249+0000 7fb2a0a15640 1 -- 172.21.15.97:0/25901902 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28c037070 con 0x7fb29c0cd280 2024-03-29T04:07:46.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.249+0000 7fdcb7fff640 1 -- 172.21.15.97:0/1563909601 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc037070 con 0x7fdcc00acf20 2024-03-29T04:07:46.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.249+0000 7fb2a0a15640 1 -- 172.21.15.97:0/25901902 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c031bc0 con 0x7fb29c0cd280 2024-03-29T04:07:46.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.249+0000 7fdcb7fff640 1 -- 172.21.15.97:0/1563909601 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc030710 con 0x7fdcc00acf20 2024-03-29T04:07:46.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.249+0000 7fc2aa960640 1 Processor -- start 2024-03-29T04:07:46.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7fdcb7fff640 1 -- 172.21.15.97:0/1563909601 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc02be60 con 0x7fdcc00acf20 2024-03-29T04:07:46.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.249+0000 7f89909ba640 1 -- 172.21.15.97:0/3148990665 wait complete. 2024-03-29T04:07:46.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7fc2aa960640 1 -- start start 2024-03-29T04:07:46.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7f7e86adb640 1 Processor -- start 2024-03-29T04:07:46.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/25901902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29c0cd280 msgr2=0x7fb29c0b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7fb2a3c7b640 1 --2- 172.21.15.97:0/25901902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29c0cd280 0x7fb29c0b30b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb29c099500 tx=0x7fb28c02b860 comp rx=0 tx=0).stop 2024-03-29T04:07:46.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7f920ec70640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92000df7f0 0x7f92000541f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7fdcc614d640 1 -- 172.21.15.97:0/1563909601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc00acf20 msgr2=0x7fdcc00ad3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7fdcc614d640 1 --2- 172.21.15.97:0/1563909601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc00acf20 0x7fdcc00ad3b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0097a0 tx=0x7fdcbc02b660 comp rx=0 tx=0).stop 2024-03-29T04:07:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7f7e86adb640 1 -- start start 2024-03-29T04:07:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7fdcc614d640 1 -- 172.21.15.97:0/1563909601 shutdown_connections 2024-03-29T04:07:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7fdcc614d640 1 --2- 172.21.15.97:0/1563909601 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc00acf20 0x7fdcc00ad3b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7fdcc614d640 1 --2- 172.21.15.97:0/1563909601 >> v2:172.21.15.97:3300/0 conn(0x7fdcc0166ea0 0x7fdcc0162f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7fdcc614d640 1 -- 172.21.15.97:0/1563909601 >> 172.21.15.97:0/1563909601 conn(0x7fdcc00b1e20 msgr2=0x7fdcc00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7f920ec70640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9200050730 0x7f9200053130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7fab58d50640 1 -- 172.21.15.97:0/2327693775 shutdown_connections 2024-03-29T04:07:46.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7f47d02d9640 1 Processor -- start 2024-03-29T04:07:46.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7f47d02d9640 1 -- start start 2024-03-29T04:07:46.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7f920ca0c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92000df7f0 0x7f92000541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7f920ca0c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92000df7f0 0x7f92000541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:35746/0 (socket says 172.21.15.97:35746) 2024-03-29T04:07:46.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7f920ca0c640 1 -- 172.21.15.97:0/698249547 learned_addr learned my addr 172.21.15.97:0/698249547 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:46.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7f89909ba640 1 Processor -- start 2024-03-29T04:07:46.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7f89909ba640 1 -- start start 2024-03-29T04:07:46.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7f920ec70640 1 -- 172.21.15.97:0/698249547 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9200052690 con 0x7f9200053670 2024-03-29T04:07:46.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7f9207fff640 1 --2- 172.21.15.97:0/698249547 >> v2:172.21.15.97:3300/0 conn(0x7f9200050730 0x7f9200053130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7f920ec70640 1 -- 172.21.15.97:0/698249547 --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f92000dfbd0 con 0x7f9200050730 2024-03-29T04:07:46.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7f920ec70640 1 -- 172.21.15.97:0/698249547 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9200053c70 con 0x7f92000df7f0 2024-03-29T04:07:46.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fdcc614d640 1 -- 172.21.15.97:0/1563909601 shutdown_connections 2024-03-29T04:07:46.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.250+0000 7fab58d50640 1 -- 172.21.15.97:0/2327693775 wait complete. 2024-03-29T04:07:46.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7f92077fe640 1 -- 172.21.15.97:0/698249547 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1014375215 0 0) 0x7f9200052690 con 0x7f9200053670 2024-03-29T04:07:46.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fc2aa960640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2a406dd90 0x7fc2a415d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7f92077fe640 1 -- 172.21.15.97:0/698249547 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91f0003520 con 0x7f9200053670 2024-03-29T04:07:46.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7f920ca0c640 1 -- 172.21.15.97:0/698249547 >> v2:172.21.15.97:3300/0 conn(0x7f9200050730 msgr2=0x7f9200053130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:46.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fc2aa960640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a4099c50 0x7fc2a40b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7f920ca0c640 1 --2- 172.21.15.97:0/698249547 >> v2:172.21.15.97:3300/0 conn(0x7f9200050730 0x7f9200053130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7f920ca0c640 1 -- 172.21.15.97:0/698249547 >> v1:172.21.15.154:6789/0 conn(0x7f9200053670 legacy=0x7f9200006710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7f920ca0c640 1 -- 172.21.15.97:0/698249547 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9200052930 con 0x7f92000df7f0 2024-03-29T04:07:46.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7f7e86adb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e800b4a50 0x7f7e801ebb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fc2aa960640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a40cd350 0x7fc2a40af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fc2aa960640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a4160720 con 0x7fc2a40cd350 2024-03-29T04:07:46.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7f9207fff640 1 --2- 172.21.15.97:0/698249547 >> v2:172.21.15.97:3300/0 conn(0x7f9200050730 0x7f9200053130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:46.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fc2aa960640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a41605a0 con 0x7fc2a4099c50 2024-03-29T04:07:46.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fc2aa960640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a40c5020 con 0x7fc2a406dd90 2024-03-29T04:07:46.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/25901902 shutdown_connections 2024-03-29T04:07:46.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fb2a3c7b640 1 --2- 172.21.15.97:0/25901902 >> v2:172.21.15.97:3300/0 conn(0x7fb29c0c5020 0x7fb29c0c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fb2a3c7b640 1 --2- 172.21.15.97:0/25901902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29c0cd280 0x7fb29c0b30b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/25901902 >> 172.21.15.97:0/25901902 conn(0x7fb29c06b510 msgr2=0x7fb29c06b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fc2a37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a4099c50 0x7fc2a40b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f47d02d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47c80b4710 0x7f47c814f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fc2a8efd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a40cd350 0x7fc2a40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7fc2a8efd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a40cd350 0x7fc2a40af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38948/0 (socket says 172.21.15.97:38948) 2024-03-29T04:07:46.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7fc2a8efd640 1 -- 172.21.15.97:0/3952377710 learned_addr learned my addr 172.21.15.97:0/3952377710 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:46.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f47d02d9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47c815c670 0x7f47c8151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f47d02d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47c8152140 0x7f47c8159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7fc2a8efd640 1 -- 172.21.15.97:0/3952377710 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2a406dd90 msgr2=0x7fc2a415d0e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:07:46.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7fc2a8efd640 1 --2- 172.21.15.97:0/3952377710 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2a406dd90 0x7fc2a415d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7fc2a8efd640 1 -- 172.21.15.97:0/3952377710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a4099c50 msgr2=0x7fc2a40b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7fc2a8efd640 1 --2- 172.21.15.97:0/3952377710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a4099c50 0x7fc2a40b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7fc2a8efd640 1 -- 172.21.15.97:0/3952377710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a415d6f0 con 0x7fc2a40cd350 2024-03-29T04:07:46.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f89909ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f898c0b5e60 0x7f898c0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f47d02d9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f47c80b3a50 con 0x7f47c8152140 2024-03-29T04:07:46.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f47d02d9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f47c80b59c0 con 0x7f47c815c670 2024-03-29T04:07:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f47ce876640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47c8152140 0x7f47c8159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f898a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f898c0b5e60 0x7f898c0c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f898a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f898c0b5e60 0x7f898c0c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55528/0 (socket says 172.21.15.97:55528) 2024-03-29T04:07:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7fc2a8efd640 1 --2- 172.21.15.97:0/3952377710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a40cd350 0x7fc2a40af910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c730 tx=0x7fc29c00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f89909ba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c0b6240 0x7f898c1f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f898a59c640 1 -- 172.21.15.97:0/3709218983 learned_addr learned my addr 172.21.15.97:0/3709218983 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:46.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/25901902 shutdown_connections 2024-03-29T04:07:46.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7fab58d50640 1 Processor -- start 2024-03-29T04:07:46.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7fab58d50640 1 -- start start 2024-03-29T04:07:46.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f89909ba640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f898c0cf440 0x7f898c15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/25901902 wait complete. 2024-03-29T04:07:46.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f89909ba640 1 -- 172.21.15.97:0/3709218983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f898c0b6650 con 0x7f898c0b6240 2024-03-29T04:07:46.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f898ad9d640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f898c0cf440 0x7f898c15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f7e85ad9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e800b4a50 0x7f7e801ebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f7e86adb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e80162d10 0x7f7e80154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f7e85ad9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e800b4a50 0x7f7e801ebb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38960/0 (socket says 172.21.15.97:38960) 2024-03-29T04:07:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f7e85ad9640 1 -- 172.21.15.97:0/1522223098 learned_addr learned my addr 172.21.15.97:0/1522223098 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f7e86adb640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e801e90e0 0x7f7e8006e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f7e86adb640 1 -- 172.21.15.97:0/1522223098 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e800cc550 con 0x7f7e800b4a50 2024-03-29T04:07:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f7e86adb640 1 -- 172.21.15.97:0/1522223098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7e800b4400 con 0x7f7e80162d10 2024-03-29T04:07:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f7e86adb640 1 -- 172.21.15.97:0/1522223098 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7e800cc6d0 con 0x7f7e801e90e0 2024-03-29T04:07:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f8989d9b640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c0b6240 0x7f898c1f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f7e852d8640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e80162d10 0x7f7e80154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f89909ba640 1 -- 172.21.15.97:0/3709218983 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f898c0d3090 con 0x7f898c0b5e60 2024-03-29T04:07:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7fc2a0ff9640 1 -- 172.21.15.97:0/3952377710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c016020 con 0x7fc2a40cd350 2024-03-29T04:07:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7fc2a0ff9640 1 -- 172.21.15.97:0/3952377710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc29c006d50 con 0x7fc2a40cd350 2024-03-29T04:07:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7fc2a0ff9640 1 -- 172.21.15.97:0/3952377710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c003b90 con 0x7fc2a40cd350 2024-03-29T04:07:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7fc2aa960640 1 -- 172.21.15.97:0/3952377710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a40ad130 con 0x7fc2a40cd350 2024-03-29T04:07:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f7e862da640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e801e90e0 0x7f7e8006e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f47ce876640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47c8152140 0x7f47c8159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38970/0 (socket says 172.21.15.97:38970) 2024-03-29T04:07:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f47ce876640 1 -- 172.21.15.97:0/3388224969 learned_addr learned my addr 172.21.15.97:0/3388224969 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7fc2aa960640 1 -- 172.21.15.97:0/3952377710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a40ad6f0 con 0x7fc2a40cd350 2024-03-29T04:07:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.251+0000 7fdcc614d640 1 -- 172.21.15.97:0/1563909601 wait complete. 2024-03-29T04:07:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f89909ba640 1 -- 172.21.15.97:0/3709218983 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f898c0b67d0 con 0x7f898c0cf440 2024-03-29T04:07:46.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f47d02d9640 1 -- 172.21.15.97:0/3388224969 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f47c80cdee0 con 0x7f47c80b4710 2024-03-29T04:07:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f47cd874640 1 --2- 172.21.15.97:0/3388224969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47c815c670 0x7f47c8151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.252+0000 7f47ce075640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47c80b4710 0x7f47c814f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7fdcc614d640 1 Processor -- start 2024-03-29T04:07:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7fab58d50640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab540b30b0 0x7fab5406ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f920ca0c640 1 --2- 172.21.15.97:0/698249547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92000df7f0 0x7f92000541f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f91f8008120 tx=0x7f91f802dd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=767f5e4c6e142968 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f92077fe640 1 -- 172.21.15.97:0/698249547 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f800b040 con 0x7f92000df7f0 2024-03-29T04:07:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f92077fe640 1 -- 172.21.15.97:0/698249547 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f8004230 con 0x7f92000df7f0 2024-03-29T04:07:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7f92077fe640 1 -- 172.21.15.97:0/698249547 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f802e910 con 0x7f92000df7f0 2024-03-29T04:07:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7fab58d50640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab540cd280 0x7fab540cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fab58d50640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab541605e0 0x7fab540cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f898a59c640 1 -- 172.21.15.97:0/3709218983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f898c0cf440 msgr2=0x7f898c15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f898a59c640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f898c0cf440 0x7f898c15dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f898a59c640 1 -- 172.21.15.97:0/3709218983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c0b6240 msgr2=0x7f898c1f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f898a59c640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c0b6240 0x7f898c1f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f898a59c640 1 -- 172.21.15.97:0/3709218983 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c1f5a10 con 0x7f898c0b5e60 2024-03-29T04:07:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fab51d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab540cd280 0x7fab540cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f7e862da640 1 -- 172.21.15.97:0/1522223098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e80162d10 msgr2=0x7f7e80154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f7e862da640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e80162d10 0x7f7e80154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f7e862da640 1 -- 172.21.15.97:0/1522223098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e800b4a50 msgr2=0x7f7e801ebb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f7e862da640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e800b4a50 0x7f7e801ebb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f7e862da640 1 -- 172.21.15.97:0/1522223098 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e801e98c0 con 0x7f7e801e90e0 2024-03-29T04:07:46.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fab51d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab540cd280 0x7fab540cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:35814/0 (socket says 172.21.15.97:35814) 2024-03-29T04:07:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f898a59c640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f898c0b5e60 0x7f898c0c9b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8984009e50 tx=0x7f8984004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fb2a3c7b640 1 Processor -- start 2024-03-29T04:07:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f920ec70640 1 -- 172.21.15.97:0/698249547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92000df7f0 msgr2=0x7f92000541f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f920ec70640 1 --2- 172.21.15.97:0/698249547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92000df7f0 0x7f92000541f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f91f8008120 tx=0x7f91f802dd60 comp rx=0 tx=0).stop 2024-03-29T04:07:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f920ec70640 1 -- 172.21.15.97:0/698249547 shutdown_connections 2024-03-29T04:07:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f920ec70640 1 --2- 172.21.15.97:0/698249547 >> v2:172.21.15.97:3300/0 conn(0x7f9200050730 0x7f9200053130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f920ec70640 1 --2- 172.21.15.97:0/698249547 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92000df7f0 0x7f92000541f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f920ec70640 1 -- 172.21.15.97:0/698249547 >> 172.21.15.97:0/698249547 conn(0x7f9200009fa0 msgr2=0x7f920000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fb2a3c7b640 1 -- start start 2024-03-29T04:07:46.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f7e862da640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e801e90e0 0x7f7e8006e630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7e78033000 tx=0x7f7e78004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fab5259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab540b30b0 0x7fab5406ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fab5259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab540b30b0 0x7fab5406ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55558/0 (socket says 172.21.15.97:55558) 2024-03-29T04:07:46.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fab58d50640 1 -- 172.21.15.97:0/1701306654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fab540ae1d0 con 0x7fab541605e0 2024-03-29T04:07:46.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fab5259c640 1 -- 172.21.15.97:0/1701306654 learned_addr learned my addr 172.21.15.97:0/1701306654 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:46.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fab58d50640 1 -- 172.21.15.97:0/1701306654 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab540ad9d0 con 0x7fab540b30b0 2024-03-29T04:07:46.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fab58d50640 1 -- 172.21.15.97:0/1701306654 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fab540ae350 con 0x7fab540cd280 2024-03-29T04:07:46.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fab52d9d640 1 --2- 172.21.15.97:0/1701306654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab541605e0 0x7fab540cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f47ce075640 1 -- 172.21.15.97:0/3388224969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47c815c670 msgr2=0x7f47c8151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f47ce075640 1 --2- 172.21.15.97:0/3388224969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47c815c670 0x7f47c8151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.253+0000 7fdcc614d640 1 -- start start 2024-03-29T04:07:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f47ce075640 1 -- 172.21.15.97:0/3388224969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47c8152140 msgr2=0x7f47c8159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f47ce075640 1 --2- 172.21.15.97:0/3388224969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47c8152140 0x7f47c8159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f47ce075640 1 -- 172.21.15.97:0/3388224969 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c8152650 con 0x7f47c80b4710 2024-03-29T04:07:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fdcc614d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc00acf20 0x7fdcc00c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fdcc614d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc00cca20 0x7fdcc00733d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fdcc614d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcc0166ea0 0x7fdcc00afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fdcc614d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0167320 con 0x7fdcc00acf20 2024-03-29T04:07:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fdcc614d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0166980 con 0x7fdcc0166ea0 2024-03-29T04:07:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7fdcc614d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc0073190 con 0x7fdcc00cca20 2024-03-29T04:07:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f47ce075640 1 --2- 172.21.15.97:0/3388224969 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47c80b4710 0x7f47c814f6e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f47c003b000 tx=0x7f47c00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fdcc494a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc00cca20 0x7fdcc00733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7f7e767fc640 1 -- 172.21.15.97:0/1522223098 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78044030 con 0x7f7e801e90e0 2024-03-29T04:07:46.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fdcc494a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc00cca20 0x7fdcc00733d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:35816/0 (socket says 172.21.15.97:35816) 2024-03-29T04:07:46.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fdcc494a640 1 -- 172.21.15.97:0/3609264812 learned_addr learned my addr 172.21.15.97:0/3609264812 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:46.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fc28a7fc640 1 -- 172.21.15.97:0/3952377710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a4075280 con 0x7fc2a40cd350 2024-03-29T04:07:46.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7f7e86adb640 1 -- 172.21.15.97:0/1522223098 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e801e9ba0 con 0x7f7e801e90e0 2024-03-29T04:07:46.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fdcc594c640 1 --2- 172.21.15.97:0/3609264812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcc0166ea0 0x7fdcc00afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7f7e86adb640 1 -- 172.21.15.97:0/1522223098 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e801ea160 con 0x7f7e801e90e0 2024-03-29T04:07:46.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7f920ec70640 1 -- 172.21.15.97:0/698249547 shutdown_connections 2024-03-29T04:07:46.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fab52d9d640 1 -- 172.21.15.97:0/1701306654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab540cd280 msgr2=0x7fab540cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fb2a3c7b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29c0c5020 0x7fb29c152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fb2a3c7b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29c0cd280 0x7fb29c15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fb2a3c7b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb29c1605e0 0x7fb29c15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7f920ec70640 1 -- 172.21.15.97:0/698249547 wait complete. 2024-03-29T04:07:46.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fb2a3c7b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c0c5990 con 0x7fb29c0cd280 2024-03-29T04:07:46.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fb2a3c7b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c0ad9d0 con 0x7fb29c0c5020 2024-03-29T04:07:46.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fb2a3c7b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb29c0c5b10 con 0x7fb29c1605e0 2024-03-29T04:07:46.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f897affd640 1 -- 172.21.15.97:0/3709218983 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984044020 con 0x7f898c0b5e60 2024-03-29T04:07:46.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fdcc494a640 1 -- 172.21.15.97:0/3609264812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcc0166ea0 msgr2=0x7fdcc00afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f897affd640 1 -- 172.21.15.97:0/3709218983 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8984040670 con 0x7f898c0b5e60 2024-03-29T04:07:46.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.254+0000 7f897affd640 1 -- 172.21.15.97:0/3709218983 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984040a60 con 0x7f898c0b5e60 2024-03-29T04:07:46.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fdcc494a640 1 --2- 172.21.15.97:0/3609264812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcc0166ea0 0x7fdcc00afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fdcc494a640 1 -- 172.21.15.97:0/3609264812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc00acf20 msgr2=0x7fdcc00c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fdcc494a640 1 --2- 172.21.15.97:0/3609264812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc00acf20 0x7fdcc00c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7f89909ba640 1 -- 172.21.15.97:0/3709218983 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898c0ca250 con 0x7f898c0b5e60 2024-03-29T04:07:46.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fab52d9d640 1 --2- 172.21.15.97:0/1701306654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab540cd280 0x7fab540cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fdcc494a640 1 -- 172.21.15.97:0/3609264812 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc01f97f0 con 0x7fdcc00cca20 2024-03-29T04:07:46.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fab52d9d640 1 -- 172.21.15.97:0/1701306654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab540b30b0 msgr2=0x7fab5406ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7f89909ba640 1 -- 172.21.15.97:0/3709218983 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898c0ca810 con 0x7f898c0b5e60 2024-03-29T04:07:46.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fab52d9d640 1 --2- 172.21.15.97:0/1701306654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab540b30b0 0x7fab5406ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fab52d9d640 1 -- 172.21.15.97:0/1701306654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab540d02e0 con 0x7fab541605e0 2024-03-29T04:07:46.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7f47beffd640 1 -- 172.21.15.97:0/3388224969 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c00396c0 con 0x7f47c80b4710 2024-03-29T04:07:46.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.255+0000 7fab52d9d640 1 --2- 172.21.15.97:0/1701306654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab541605e0 0x7fab540cfd70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab4c033000 tx=0x7fab4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.256+0000 7fb2a1a17640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29c0c5020 0x7fb29c152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.256+0000 7fb2a1a17640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29c0c5020 0x7fb29c152830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55580/0 (socket says 172.21.15.97:55580) 2024-03-29T04:07:46.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.256+0000 7fb2a1a17640 1 -- 172.21.15.97:0/966627393 learned_addr learned my addr 172.21.15.97:0/966627393 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:46.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.256+0000 7f47d02d9640 1 -- 172.21.15.97:0/3388224969 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c8152930 con 0x7f47c80b4710 2024-03-29T04:07:46.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.256+0000 7f47d02d9640 1 -- 172.21.15.97:0/3388224969 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c80ce4d0 con 0x7f47c80b4710 2024-03-29T04:07:46.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.256+0000 7fb2a2218640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb29c1605e0 0x7fb29c15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.257+0000 7fb2a1216640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29c0cd280 0x7fb29c15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.256+0000 7f7e767fc640 1 -- 172.21.15.97:0/1522223098 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e78035d30 con 0x7f7e801e90e0 2024-03-29T04:07:46.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.256+0000 7f7e767fc640 1 -- 172.21.15.97:0/1522223098 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78039040 con 0x7f7e801e90e0 2024-03-29T04:07:46.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.256+0000 7f47beffd640 1 -- 172.21.15.97:0/3388224969 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c003bcc0 con 0x7f47c80b4710 2024-03-29T04:07:46.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.257+0000 7f920ec70640 1 Processor -- start 2024-03-29T04:07:46.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.256+0000 7fdcc494a640 1 --2- 172.21.15.97:0/3609264812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc00cca20 0x7fdcc00733d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800efc0 tx=0x7fdcb800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.257+0000 7fdcb5ffb640 1 -- 172.21.15.97:0/3609264812 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8018020 con 0x7fdcc00cca20 2024-03-29T04:07:46.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.257+0000 7fdcb5ffb640 1 -- 172.21.15.97:0/3609264812 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb8014480 con 0x7fdcc00cca20 2024-03-29T04:07:46.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.257+0000 7f89909ba640 1 -- 172.21.15.97:0/3709218983 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898c04f8f0 con 0x7f898c0b5e60 2024-03-29T04:07:46.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.257+0000 7fb2a2218640 1 -- 172.21.15.97:0/966627393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29c0c5020 msgr2=0x7fb29c152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.257+0000 7fb2a2218640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29c0c5020 0x7fb29c152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.257+0000 7fb2a2218640 1 -- 172.21.15.97:0/966627393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29c0cd280 msgr2=0x7fb29c15e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.257+0000 7fb2a2218640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29c0cd280 0x7fb29c15e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.257+0000 7f47beffd640 1 -- 172.21.15.97:0/3388224969 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0046660 con 0x7f47c80b4710 2024-03-29T04:07:46.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.257+0000 7fb2a2218640 1 -- 172.21.15.97:0/966627393 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c152d70 con 0x7fb29c1605e0 2024-03-29T04:07:46.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.258+0000 7fb2a1a17640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29c0c5020 0x7fb29c152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:46.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.258+0000 7f47d02d9640 1 -- 172.21.15.97:0/3388224969 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c8068d50 con 0x7f47c80b4710 2024-03-29T04:07:46.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.257+0000 7f7e63fff640 1 -- 172.21.15.97:0/1522223098 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e8004f8f0 con 0x7f7e801e90e0 2024-03-29T04:07:46.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.258+0000 7fb2a2218640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb29c1605e0 0x7fb29c15e9d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb29400ce20 tx=0x7fb29400a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.257+0000 7f920ec70640 1 -- start start 2024-03-29T04:07:46.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.258+0000 7fdcc614d640 1 -- 172.21.15.97:0/3609264812 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc00c6610 con 0x7fdcc00cca20 2024-03-29T04:07:46.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.258+0000 7fdcb5ffb640 1 -- 172.21.15.97:0/3609264812 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8014850 con 0x7fdcc00cca20 2024-03-29T04:07:46.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.258+0000 7f47beffd640 1 -- 172.21.15.97:0/3388224969 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f47c004f020 con 0x7f47c80b4710 2024-03-29T04:07:46.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.258+0000 7fc2a0ff9640 1 -- 172.21.15.97:0/3952377710 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc29c0050d0 con 0x7fc2a40cd350 2024-03-29T04:07:46.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.258+0000 7fdcc614d640 1 -- 172.21.15.97:0/3609264812 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc00c6bd0 con 0x7fdcc00cca20 2024-03-29T04:07:46.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.258+0000 7f897affd640 1 -- 172.21.15.97:0/3709218983 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8984047020 con 0x7f898c0b5e60 2024-03-29T04:07:46.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.258+0000 7f47beffd640 1 --2- 172.21.15.97:0/3388224969 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f47ac06c630 0x7f47ac06eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f47beffd640 1 -- 172.21.15.97:0/3388224969 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f47c00b62d0 con 0x7f47c80b4710 2024-03-29T04:07:46.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f920ec70640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9200050730 0x7f92000d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f920ec70640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9200053670 0x7f92000dfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f920ec70640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92000df7f0 0x7f92000cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f920ec70640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9200053f50 con 0x7f9200050730 2024-03-29T04:07:46.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7fab42ffd640 1 -- 172.21.15.97:0/1701306654 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c044020 con 0x7fab541605e0 2024-03-29T04:07:46.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f920ec70640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9200054840 con 0x7f9200053670 2024-03-29T04:07:46.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7fc2a0ff9640 1 --2- 172.21.15.97:0/3952377710 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc29006c600 0x7fc29006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7fc2a0ff9640 1 -- 172.21.15.97:0/3952377710 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc29c08c140 con 0x7fc2a40cd350 2024-03-29T04:07:46.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f920ec70640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9200006e20 con 0x7f92000df7f0 2024-03-29T04:07:46.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f9207fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9200053670 0x7f92000dfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f47cd874640 1 --2- 172.21.15.97:0/3388224969 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f47ac06c630 0x7f47ac06eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f9207fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9200053670 0x7f92000dfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55596/0 (socket says 172.21.15.97:55596) 2024-03-29T04:07:46.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f9207fff640 1 -- 172.21.15.97:0/810498500 learned_addr learned my addr 172.21.15.97:0/810498500 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:46.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7fc2a3fff640 1 --2- 172.21.15.97:0/3952377710 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc29006c600 0x7fc29006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f920d20d640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92000df7f0 0x7f92000cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f920ca0c640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9200050730 0x7f92000d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7fdcc614d640 1 -- 172.21.15.97:0/3609264812 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc004f8f0 con 0x7fdcc00cca20 2024-03-29T04:07:46.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7f897affd640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f896c06c580 0x7f896c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7fab58d50640 1 -- 172.21.15.97:0/1701306654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab540ceb80 con 0x7fab541605e0 2024-03-29T04:07:46.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.259+0000 7fab58d50640 1 -- 172.21.15.97:0/1701306654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab540cf3c0 con 0x7fab541605e0 2024-03-29T04:07:46.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.260+0000 7f8989d9b640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f896c06c580 0x7f896c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.260+0000 7f897affd640 1 -- 172.21.15.97:0/3709218983 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f89840ae100 con 0x7f898c0b5e60 2024-03-29T04:07:46.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.260+0000 7f920d20d640 1 -- 172.21.15.97:0/810498500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9200053670 msgr2=0x7f92000dfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.260+0000 7f920d20d640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9200053670 0x7f92000dfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.260+0000 7f920d20d640 1 -- 172.21.15.97:0/810498500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9200050730 msgr2=0x7f92000d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.260+0000 7f920d20d640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9200050730 0x7f92000d32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.260+0000 7f920d20d640 1 -- 172.21.15.97:0/810498500 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92000e05a0 con 0x7f92000df7f0 2024-03-29T04:07:46.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.260+0000 7f8989d9b640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f896c06c580 0x7f896c06ea40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f898c0d5450 tx=0x7f897c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.260+0000 7f920d20d640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92000df7f0 0x7f92000cfb40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f92080510d0 tx=0x7f920806da70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.260+0000 7fc2a3fff640 1 --2- 172.21.15.97:0/3952377710 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc29006c600 0x7fc29006eac0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc298009670 tx=0x7fc298006bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.260+0000 7f7e767fc640 1 -- 172.21.15.97:0/1522223098 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7e78047070 con 0x7f7e801e90e0 2024-03-29T04:07:46.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.261+0000 7f7e767fc640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7e6406c5b0 0x7f7e6406ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.260+0000 7f47cd874640 1 --2- 172.21.15.97:0/3388224969 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f47ac06c630 0x7f47ac06eaf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47c815b2c0 tx=0x7f47c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.261+0000 7f7e85ad9640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7e6406c5b0 0x7f7e6406ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.261+0000 7f7e767fc640 1 -- 172.21.15.97:0/1522223098 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e780adc30 con 0x7f7e801e90e0 2024-03-29T04:07:46.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.261+0000 7f7e85ad9640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7e6406c5b0 0x7f7e6406ea70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009370 tx=0x7f7e70009300 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.261+0000 7fab58d50640 1 -- 172.21.15.97:0/1701306654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab54068640 con 0x7fab541605e0 2024-03-29T04:07:46.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.263+0000 7f92057fa640 1 -- 172.21.15.97:0/810498500 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208073360 con 0x7f92000df7f0 2024-03-29T04:07:46.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.263+0000 7f920ec70640 1 -- 172.21.15.97:0/810498500 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92000e0880 con 0x7f92000df7f0 2024-03-29T04:07:46.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.263+0000 7f920ec70640 1 -- 172.21.15.97:0/810498500 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f920004c890 con 0x7f92000df7f0 2024-03-29T04:07:46.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.263+0000 7f47beffd640 1 -- 172.21.15.97:0/3388224969 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f47c0056020 con 0x7f47c80b4710 2024-03-29T04:07:46.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.263+0000 7fb2927fc640 1 -- 172.21.15.97:0/966627393 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29401a070 con 0x7fb29c1605e0 2024-03-29T04:07:46.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.263+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/966627393 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29c0afbe0 con 0x7fb29c1605e0 2024-03-29T04:07:46.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.263+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/966627393 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29c0b01a0 con 0x7fb29c1605e0 2024-03-29T04:07:46.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.263+0000 7fc2a0ff9640 1 -- 172.21.15.97:0/3952377710 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc29c020020 con 0x7fc2a40cd350 2024-03-29T04:07:46.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.264+0000 7fab42ffd640 1 -- 172.21.15.97:0/1701306654 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab4c03a040 con 0x7fab541605e0 2024-03-29T04:07:46.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.264+0000 7f897affd640 1 -- 172.21.15.97:0/3709218983 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f898404e030 con 0x7f898c0b5e60 2024-03-29T04:07:46.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.264+0000 7f920ec70640 1 -- 172.21.15.97:0/810498500 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9200005380 con 0x7f92000df7f0 2024-03-29T04:07:46.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.265+0000 7fab42ffd640 1 -- 172.21.15.97:0/1701306654 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c039070 con 0x7fab541605e0 2024-03-29T04:07:46.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.265+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/966627393 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb29c068640 con 0x7fb29c1605e0 2024-03-29T04:07:46.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.266+0000 7fab42ffd640 1 -- 172.21.15.97:0/1701306654 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fab4c047020 con 0x7fab541605e0 2024-03-29T04:07:46.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.266+0000 7fab42ffd640 1 --2- 172.21.15.97:0/1701306654 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fab3406c530 0x7fab3406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.267+0000 7fab42ffd640 1 -- 172.21.15.97:0/1701306654 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fab4c0ae040 con 0x7fab541605e0 2024-03-29T04:07:46.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.267+0000 7fab5259c640 1 --2- 172.21.15.97:0/1701306654 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fab3406c530 0x7fab3406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.267+0000 7fab5259c640 1 --2- 172.21.15.97:0/1701306654 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fab3406c530 0x7fab3406e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fab44001eb0 tx=0x7fab44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.268+0000 7fb2927fc640 1 -- 172.21.15.97:0/966627393 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb29400d840 con 0x7fb29c1605e0 2024-03-29T04:07:46.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.268+0000 7fb2927fc640 1 -- 172.21.15.97:0/966627393 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294024060 con 0x7fb29c1605e0 2024-03-29T04:07:46.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.268+0000 7f92057fa640 1 -- 172.21.15.97:0/810498500 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9208073500 con 0x7f92000df7f0 2024-03-29T04:07:46.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.268+0000 7f7e767fc640 1 -- 172.21.15.97:0/1522223098 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7e78047360 con 0x7f7e801e90e0 2024-03-29T04:07:46.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.269+0000 7f92057fa640 1 -- 172.21.15.97:0/810498500 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208099060 con 0x7f92000df7f0 2024-03-29T04:07:46.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.269+0000 7fdcb5ffb640 1 -- 172.21.15.97:0/3609264812 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdcb8005d10 con 0x7fdcc00cca20 2024-03-29T04:07:46.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.270+0000 7fdcb5ffb640 1 --2- 172.21.15.97:0/3609264812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdca406c600 0x7fdca406eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.270+0000 7fab42ffd640 1 -- 172.21.15.97:0/1701306654 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fab4c04e030 con 0x7fab541605e0 2024-03-29T04:07:46.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.270+0000 7fdcb5ffb640 1 -- 172.21.15.97:0/3609264812 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdcb808cc80 con 0x7fdcc00cca20 2024-03-29T04:07:46.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.271+0000 7fdcc514b640 1 --2- 172.21.15.97:0/3609264812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdca406c600 0x7fdca406eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.271+0000 7fdcc514b640 1 --2- 172.21.15.97:0/3609264812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdca406c600 0x7fdca406eac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0007fd0 tx=0x7fdcb0007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.271+0000 7f92057fa640 1 -- 172.21.15.97:0/810498500 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9208073730 con 0x7f92000df7f0 2024-03-29T04:07:46.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.271+0000 7f92057fa640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f91f406c530 0x7f91f406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.271+0000 7f920ca0c640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f91f406c530 0x7f91f406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.271+0000 7fb2927fc640 1 -- 172.21.15.97:0/966627393 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb29401d070 con 0x7fb29c1605e0 2024-03-29T04:07:46.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.271+0000 7fb2927fc640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb28006c530 0x7fb28006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:46.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.272+0000 7fb2a1a17640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb28006c530 0x7fb28006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:46.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.272+0000 7fb2a1a17640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb28006c530 0x7fb28006e9f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb298005880 tx=0x7fb298009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.272+0000 7f920ca0c640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f91f406c530 0x7f91f406e9f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f91f8037000 tx=0x7f91f800c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:46.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.273+0000 7fb2927fc640 1 -- 172.21.15.97:0/966627393 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb294058370 con 0x7fb29c1605e0 2024-03-29T04:07:46.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.273+0000 7fdcb5ffb640 1 -- 172.21.15.97:0/3609264812 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdcb8022020 con 0x7fdcc00cca20 2024-03-29T04:07:46.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.276+0000 7f92057fa640 1 -- 172.21.15.97:0/810498500 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f92080d2720 con 0x7f92000df7f0 2024-03-29T04:07:46.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.278+0000 7fb2927fc640 1 -- 172.21.15.97:0/966627393 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb29405a2d0 con 0x7fb29c1605e0 2024-03-29T04:07:46.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.283+0000 7f92057fa640 1 -- 172.21.15.97:0/810498500 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f92080d4680 con 0x7f92000df7f0 2024-03-29T04:07:46.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.671+0000 7f7e63fff640 1 -- 172.21.15.97:0/1522223098 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7e800b2e90 con 0x7f7e801e90e0 2024-03-29T04:07:46.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.671+0000 7f7e767fc640 1 -- 172.21.15.97:0/1522223098 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7e7807c5a0 con 0x7f7e801e90e0 2024-03-29T04:07:46.651 INFO:teuthology.orchestra.run.smithi097.stdout:55834574853 2024-03-29T04:07:46.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.674+0000 7f7e86adb640 1 -- 172.21.15.97:0/1522223098 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7e6406c5b0 msgr2=0x7f7e6406ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.675+0000 7f7e86adb640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7e6406c5b0 0x7f7e6406ea70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009370 tx=0x7f7e70009300 comp rx=0 tx=0).stop 2024-03-29T04:07:46.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.675+0000 7f7e86adb640 1 -- 172.21.15.97:0/1522223098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e801e90e0 msgr2=0x7f7e8006e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.675+0000 7f7e86adb640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e801e90e0 0x7f7e8006e630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7e78033000 tx=0x7f7e78004a40 comp rx=0 tx=0).stop 2024-03-29T04:07:46.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.675+0000 7f7e86adb640 1 -- 172.21.15.97:0/1522223098 shutdown_connections 2024-03-29T04:07:46.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.675+0000 7f7e86adb640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7e6406c5b0 0x7f7e6406ea70 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009370 tx=0x7f7e70009300 comp rx=0 tx=0).stop 2024-03-29T04:07:46.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.675+0000 7f7e86adb640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e801e90e0 0x7f7e8006e630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.675+0000 7f7e86adb640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e80162d10 0x7f7e80154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.675+0000 7fc28a7fc640 1 -- 172.21.15.97:0/3952377710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc2a4068640 con 0x7fc2a40cd350 2024-03-29T04:07:46.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.675+0000 7f7e86adb640 1 --2- 172.21.15.97:0/1522223098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e800b4a50 0x7f7e801ebb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.675+0000 7f7e86adb640 1 -- 172.21.15.97:0/1522223098 >> 172.21.15.97:0/1522223098 conn(0x7f7e800b1cb0 msgr2=0x7f7e800723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.676+0000 7f7e86adb640 1 -- 172.21.15.97:0/1522223098 shutdown_connections 2024-03-29T04:07:46.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.676+0000 7f7e86adb640 1 -- 172.21.15.97:0/1522223098 wait complete. 2024-03-29T04:07:46.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.676+0000 7fc2a0ff9640 1 -- 172.21.15.97:0/3952377710 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc29c05a7d0 con 0x7fc2a40cd350 2024-03-29T04:07:46.657 INFO:teuthology.orchestra.run.smithi097.stdout:55834574853 2024-03-29T04:07:46.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.679+0000 7fc2aa960640 1 -- 172.21.15.97:0/3952377710 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc29006c600 msgr2=0x7fc29006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.679+0000 7fc2aa960640 1 --2- 172.21.15.97:0/3952377710 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc29006c600 0x7fc29006eac0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc298009670 tx=0x7fc298006bd0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.679+0000 7fc2aa960640 1 -- 172.21.15.97:0/3952377710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a40cd350 msgr2=0x7fc2a40af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.679+0000 7fc2aa960640 1 --2- 172.21.15.97:0/3952377710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a40cd350 0x7fc2a40af910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c730 tx=0x7fc29c00cb30 comp rx=0 tx=0).stop 2024-03-29T04:07:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.680+0000 7fc2aa960640 1 -- 172.21.15.97:0/3952377710 shutdown_connections 2024-03-29T04:07:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.680+0000 7fc2aa960640 1 --2- 172.21.15.97:0/3952377710 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc29006c600 0x7fc29006eac0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.680+0000 7fc2aa960640 1 --2- 172.21.15.97:0/3952377710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2a40cd350 0x7fc2a40af910 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.680+0000 7fc2aa960640 1 --2- 172.21.15.97:0/3952377710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2a4099c50 0x7fc2a40b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.680+0000 7fc2aa960640 1 --2- 172.21.15.97:0/3952377710 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc2a406dd90 0x7fc2a415d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.680+0000 7fc2aa960640 1 -- 172.21.15.97:0/3952377710 >> 172.21.15.97:0/3952377710 conn(0x7fc2a4072ba0 msgr2=0x7fc2a415e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.680+0000 7fc2aa960640 1 -- 172.21.15.97:0/3952377710 shutdown_connections 2024-03-29T04:07:46.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.681+0000 7fc2aa960640 1 -- 172.21.15.97:0/3952377710 wait complete. 2024-03-29T04:07:46.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.684+0000 7f89909ba640 1 -- 172.21.15.97:0/3709218983 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f898c0cf220 con 0x7f898c0b5e60 2024-03-29T04:07:46.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.684+0000 7f897affd640 1 -- 172.21.15.97:0/3709218983 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f898404e030 con 0x7f898c0b5e60 2024-03-29T04:07:46.664 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2024-03-29T04:07:46.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.687+0000 7f47d02d9640 1 -- 172.21.15.97:0/3388224969 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f47c81e5dd0 con 0x7f47c80b4710 2024-03-29T04:07:46.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.687+0000 7f47beffd640 1 -- 172.21.15.97:0/3388224969 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f47c0084980 con 0x7f47c80b4710 2024-03-29T04:07:46.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.688+0000 7f8978ff9640 1 -- 172.21.15.97:0/3709218983 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f896c06c580 msgr2=0x7f896c06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.688+0000 7f8978ff9640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f896c06c580 0x7f896c06ea40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f898c0d5450 tx=0x7f897c009040 comp rx=0 tx=0).stop 2024-03-29T04:07:46.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.688+0000 7f8978ff9640 1 -- 172.21.15.97:0/3709218983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f898c0b5e60 msgr2=0x7f898c0c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.688+0000 7f8978ff9640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f898c0b5e60 0x7f898c0c9b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8984009e50 tx=0x7f8984004a40 comp rx=0 tx=0).stop 2024-03-29T04:07:46.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.688+0000 7f8978ff9640 1 -- 172.21.15.97:0/3709218983 shutdown_connections 2024-03-29T04:07:46.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.688+0000 7f8978ff9640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f896c06c580 0x7f896c06ea40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.688+0000 7f8978ff9640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f898c0cf440 0x7f898c15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.688+0000 7f8978ff9640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c0b6240 0x7f898c1f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.688+0000 7f8978ff9640 1 --2- 172.21.15.97:0/3709218983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f898c0b5e60 0x7f898c0c9b50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.688+0000 7f8978ff9640 1 -- 172.21.15.97:0/3709218983 >> 172.21.15.97:0/3709218983 conn(0x7f898c0d0e60 msgr2=0x7f898c06c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.688+0000 7f8978ff9640 1 -- 172.21.15.97:0/3709218983 shutdown_connections 2024-03-29T04:07:46.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.688+0000 7f8978ff9640 1 -- 172.21.15.97:0/3709218983 wait complete. 2024-03-29T04:07:46.671 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2024-03-29T04:07:46.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.692+0000 7f47d02d9640 1 -- 172.21.15.97:0/3388224969 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f47ac06c630 msgr2=0x7f47ac06eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.692+0000 7f47d02d9640 1 --2- 172.21.15.97:0/3388224969 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f47ac06c630 0x7f47ac06eaf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47c815b2c0 tx=0x7f47c4009040 comp rx=0 tx=0).stop 2024-03-29T04:07:46.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.692+0000 7f47d02d9640 1 -- 172.21.15.97:0/3388224969 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47c80b4710 msgr2=0x7f47c814f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.692+0000 7f47d02d9640 1 --2- 172.21.15.97:0/3388224969 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47c80b4710 0x7f47c814f6e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f47c003b000 tx=0x7f47c00046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.692+0000 7f47d02d9640 1 -- 172.21.15.97:0/3388224969 shutdown_connections 2024-03-29T04:07:46.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.692+0000 7f47d02d9640 1 --2- 172.21.15.97:0/3388224969 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f47ac06c630 0x7f47ac06eaf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.692+0000 7f47d02d9640 1 --2- 172.21.15.97:0/3388224969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47c8152140 0x7f47c8159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.692+0000 7f47d02d9640 1 --2- 172.21.15.97:0/3388224969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47c815c670 0x7f47c8151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.692+0000 7f47d02d9640 1 --2- 172.21.15.97:0/3388224969 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47c80b4710 0x7f47c814f6e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.692+0000 7f47d02d9640 1 -- 172.21.15.97:0/3388224969 >> 172.21.15.97:0/3388224969 conn(0x7f47c8072d70 msgr2=0x7f47c806ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.692+0000 7f47d02d9640 1 -- 172.21.15.97:0/3388224969 shutdown_connections 2024-03-29T04:07:46.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.692+0000 7f47d02d9640 1 -- 172.21.15.97:0/3388224969 wait complete. 2024-03-29T04:07:46.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.692+0000 7fdcc614d640 1 -- 172.21.15.97:0/3609264812 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdcc006ee40 con 0x7fdcc00cca20 2024-03-29T04:07:46.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.693+0000 7fdcb5ffb640 1 -- 172.21.15.97:0/3609264812 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdcb8010040 con 0x7fdcc00cca20 2024-03-29T04:07:46.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.693+0000 7fab58d50640 1 -- 172.21.15.97:0/1701306654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fab540ceea0 con 0x7fab541605e0 2024-03-29T04:07:46.674 INFO:teuthology.orchestra.run.smithi097.stdout:55834574853 2024-03-29T04:07:46.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-03-29T04:07:46.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.693+0000 7fab42ffd640 1 -- 172.21.15.97:0/1701306654 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fab540ceb80 con 0x7fab541605e0 2024-03-29T04:07:46.675 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2024-03-29T04:07:46.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fdcc614d640 1 -- 172.21.15.97:0/3609264812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdca406c600 msgr2=0x7fdca406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fdcc614d640 1 --2- 172.21.15.97:0/3609264812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdca406c600 0x7fdca406eac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0007fd0 tx=0x7fdcb0007980 comp rx=0 tx=0).stop 2024-03-29T04:07:46.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fdcc614d640 1 -- 172.21.15.97:0/3609264812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc00cca20 msgr2=0x7fdcc00733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fdcc614d640 1 --2- 172.21.15.97:0/3609264812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc00cca20 0x7fdcc00733d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800efc0 tx=0x7fdcb800c490 comp rx=0 tx=0).stop 2024-03-29T04:07:46.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fab58d50640 1 -- 172.21.15.97:0/1701306654 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fab3406c530 msgr2=0x7fab3406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fab58d50640 1 --2- 172.21.15.97:0/1701306654 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fab3406c530 0x7fab3406e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fab44001eb0 tx=0x7fab44009040 comp rx=0 tx=0).stop 2024-03-29T04:07:46.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fab58d50640 1 -- 172.21.15.97:0/1701306654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab541605e0 msgr2=0x7fab540cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fab58d50640 1 --2- 172.21.15.97:0/1701306654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab541605e0 0x7fab540cfd70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab4c033000 tx=0x7fab4c004a40 comp rx=0 tx=0).stop 2024-03-29T04:07:46.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fab58d50640 1 -- 172.21.15.97:0/1701306654 shutdown_connections 2024-03-29T04:07:46.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fab58d50640 1 --2- 172.21.15.97:0/1701306654 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fab3406c530 0x7fab3406e9f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fab58d50640 1 --2- 172.21.15.97:0/1701306654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab541605e0 0x7fab540cfd70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fab58d50640 1 --2- 172.21.15.97:0/1701306654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fab540cd280 0x7fab540cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fdcc614d640 1 -- 172.21.15.97:0/3609264812 shutdown_connections 2024-03-29T04:07:46.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fdcc614d640 1 --2- 172.21.15.97:0/3609264812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdca406c600 0x7fdca406eac0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fdcc614d640 1 --2- 172.21.15.97:0/3609264812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcc0166ea0 0x7fdcc00afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fab58d50640 1 --2- 172.21.15.97:0/1701306654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab540b30b0 0x7fab5406ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fdcc614d640 1 --2- 172.21.15.97:0/3609264812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc00cca20 0x7fdcc00733d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fab58d50640 1 -- 172.21.15.97:0/1701306654 >> 172.21.15.97:0/1701306654 conn(0x7fab5406b510 msgr2=0x7fab54165420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fdcc614d640 1 --2- 172.21.15.97:0/3609264812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc00acf20 0x7fdcc00c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fdcc614d640 1 -- 172.21.15.97:0/3609264812 >> 172.21.15.97:0/3609264812 conn(0x7fdcc00b1e20 msgr2=0x7fdcc00cac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fab58d50640 1 -- 172.21.15.97:0/1701306654 shutdown_connections 2024-03-29T04:07:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fdcc614d640 1 -- 172.21.15.97:0/3609264812 shutdown_connections 2024-03-29T04:07:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fdcc614d640 1 -- 172.21.15.97:0/3609264812 wait complete. 2024-03-29T04:07:46.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.697+0000 7fab58d50640 1 -- 172.21.15.97:0/1701306654 wait complete. 2024-03-29T04:07:46.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.698+0000 7f920ec70640 1 -- 172.21.15.97:0/810498500 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f92000074e0 con 0x7f92000df7f0 2024-03-29T04:07:46.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.698+0000 7f92057fa640 1 -- 172.21.15.97:0/810498500 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f92080d8980 con 0x7f92000df7f0 2024-03-29T04:07:46.682 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2024-03-29T04:07:46.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.699+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/966627393 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb29c0af750 con 0x7fb29c1605e0 2024-03-29T04:07:46.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.699+0000 7fb2927fc640 1 -- 172.21.15.97:0/966627393 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb29405e5d0 con 0x7fb29c1605e0 2024-03-29T04:07:46.683 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-03-29T04:07:46.683 INFO:teuthology.orchestra.run.smithi097.stdout:55834574853 2024-03-29T04:07:46.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.701+0000 7f920ec70640 1 -- 172.21.15.97:0/810498500 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f91f406c530 msgr2=0x7f91f406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.701+0000 7f920ec70640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f91f406c530 0x7f91f406e9f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f91f8037000 tx=0x7f91f800c040 comp rx=0 tx=0).stop 2024-03-29T04:07:46.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.701+0000 7f920ec70640 1 -- 172.21.15.97:0/810498500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92000df7f0 msgr2=0x7f92000cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.701+0000 7f920ec70640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92000df7f0 0x7f92000cfb40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f92080510d0 tx=0x7f920806da70 comp rx=0 tx=0).stop 2024-03-29T04:07:46.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.702+0000 7f920ec70640 1 -- 172.21.15.97:0/810498500 shutdown_connections 2024-03-29T04:07:46.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.702+0000 7f920ec70640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f91f406c530 0x7f91f406e9f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.702+0000 7f920ec70640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f92000df7f0 0x7f92000cfb40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.702+0000 7f920ec70640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9200053670 0x7f92000dfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.702+0000 7f920ec70640 1 --2- 172.21.15.97:0/810498500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9200050730 0x7f92000d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.702+0000 7f920ec70640 1 -- 172.21.15.97:0/810498500 >> 172.21.15.97:0/810498500 conn(0x7f9200009fa0 msgr2=0x7f92000dce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.702+0000 7f920ec70640 1 -- 172.21.15.97:0/810498500 shutdown_connections 2024-03-29T04:07:46.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.702+0000 7f920ec70640 1 -- 172.21.15.97:0/810498500 wait complete. 2024-03-29T04:07:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.703+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/966627393 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb28006c530 msgr2=0x7fb28006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.703+0000 7fb2a3c7b640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb28006c530 0x7fb28006e9f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb298005880 tx=0x7fb298009040 comp rx=0 tx=0).stop 2024-03-29T04:07:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.703+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/966627393 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb29c1605e0 msgr2=0x7fb29c15e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.703+0000 7fb2a3c7b640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb29c1605e0 0x7fb29c15e9d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb29400ce20 tx=0x7fb29400a710 comp rx=0 tx=0).stop 2024-03-29T04:07:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.704+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/966627393 shutdown_connections 2024-03-29T04:07:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.704+0000 7fb2a3c7b640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb28006c530 0x7fb28006e9f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.704+0000 7fb2a3c7b640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb29c1605e0 0x7fb29c15e9d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.704+0000 7fb2a3c7b640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29c0cd280 0x7fb29c15e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.704+0000 7fb2a3c7b640 1 --2- 172.21.15.97:0/966627393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29c0c5020 0x7fb29c152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.704+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/966627393 >> 172.21.15.97:0/966627393 conn(0x7fb29c06b510 msgr2=0x7fb29c162500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:46.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.704+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/966627393 shutdown_connections 2024-03-29T04:07:46.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:46.704+0000 7fb2a3c7b640 1 -- 172.21.15.97:0/966627393 wait complete. 2024-03-29T04:07:46.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-03-29T04:07:46.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-03-29T04:07:46.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-03-29T04:07:46.698 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-03-29T04:07:46.699 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-03-29T04:07:46.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-03-29T04:07:47.676 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-29T04:07:47.684 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-29T04:07:47.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-29T04:07:47.694 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-29T04:07:47.697 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-29T04:07:47.699 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-29T04:07:47.700 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-29T04:07:47.700 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-29T04:07:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.860+0000 7fed76d15640 1 Processor -- start 2024-03-29T04:07:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.860+0000 7fed76d15640 1 -- start start 2024-03-29T04:07:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.861+0000 7fed76d15640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed68009180 0x7fed68052ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.861+0000 7fed76d15640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fed68049060 0x7fed680eb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.861+0000 7fed76d15640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed68009560 con 0x7fed68048c30 2024-03-29T04:07:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.861+0000 7fed76d15640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fed68035960 con 0x7fed68009180 2024-03-29T04:07:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.861+0000 7fed76d15640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fed68035c20 con 0x7fed68049060 2024-03-29T04:07:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.862+0000 7fed752b2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fed68049060 0x7fed680eb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.862+0000 7fed752b2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fed68049060 0x7fed680eb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55598/0 (socket says 172.21.15.97:55598) 2024-03-29T04:07:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.862+0000 7fed752b2640 1 -- 172.21.15.97:0/466001933 learned_addr learned my addr 172.21.15.97:0/466001933 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.862+0000 7fed752b2640 1 -- 172.21.15.97:0/466001933 >> v2:172.21.15.97:3300/0 conn(0x7fed68009180 msgr2=0x7fed68052ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:07:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.862+0000 7fed752b2640 1 --2- 172.21.15.97:0/466001933 >> v2:172.21.15.97:3300/0 conn(0x7fed68009180 0x7fed68052ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.863+0000 7fed752b2640 1 -- 172.21.15.97:0/466001933 >> v1:172.21.15.97:6789/0 conn(0x7fed68048c30 legacy=0x7fed680eb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.863+0000 7fed752b2640 1 -- 172.21.15.97:0/466001933 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed68035f00 con 0x7fed68049060 2024-03-29T04:07:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.864+0000 7fed752b2640 1 --2- 172.21.15.97:0/466001933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed68049060 0x7fed680eb9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed7002d940 tx=0x7fed700a72a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4492b8d3e6fef492 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.864+0000 7fed6f7fe640 1 -- 172.21.15.97:0/466001933 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700b4070 con 0x7fed68049060 2024-03-29T04:07:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.864+0000 7fed6f7fe640 1 -- 172.21.15.97:0/466001933 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed700af680 con 0x7fed68049060 2024-03-29T04:07:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.865+0000 7fed76d15640 1 -- 172.21.15.97:0/466001933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed68049060 msgr2=0x7fed680eb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.865+0000 7fed76d15640 1 --2- 172.21.15.97:0/466001933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed68049060 0x7fed680eb9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed7002d940 tx=0x7fed700a72a0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.866+0000 7fed76d15640 1 -- 172.21.15.97:0/466001933 shutdown_connections 2024-03-29T04:07:47.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.866+0000 7fed76d15640 1 --2- 172.21.15.97:0/466001933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed68049060 0x7fed680eb9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.866+0000 7fed76d15640 1 --2- 172.21.15.97:0/466001933 >> v2:172.21.15.97:3300/0 conn(0x7fed68009180 0x7fed68052ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.866+0000 7fed76d15640 1 -- 172.21.15.97:0/466001933 >> 172.21.15.97:0/466001933 conn(0x7fed6801ef30 msgr2=0x7fed68055c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:47.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.866+0000 7fed76d15640 1 -- 172.21.15.97:0/466001933 shutdown_connections 2024-03-29T04:07:47.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.866+0000 7fed76d15640 1 -- 172.21.15.97:0/466001933 wait complete. 2024-03-29T04:07:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.868+0000 7fed76d15640 1 Processor -- start 2024-03-29T04:07:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.868+0000 7fed76d15640 1 -- start start 2024-03-29T04:07:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.869+0000 7fed76d15640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed68009180 0x7fed68050ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.870+0000 7fed76d15640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed68048c30 0x7fed68033ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.870+0000 7fed76d15640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed680e7d50 0x7fed680e8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.870+0000 7fed76d15640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fed680367a0 con 0x7fed68009180 2024-03-29T04:07:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.870+0000 7fed76d15640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fed680364c0 con 0x7fed68048c30 2024-03-29T04:07:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.870+0000 7fed76d15640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fed68054090 con 0x7fed680e7d50 2024-03-29T04:07:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.870+0000 7fed752b2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed680e7d50 0x7fed680e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.870+0000 7fed752b2640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed680e7d50 0x7fed680e8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:35842/0 (socket says 172.21.15.97:35842) 2024-03-29T04:07:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.870+0000 7fed752b2640 1 -- 172.21.15.97:0/465494236 learned_addr learned my addr 172.21.15.97:0/465494236 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.870+0000 7fed6ffff640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed68048c30 0x7fed68033ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.870+0000 7fed752b2640 1 -- 172.21.15.97:0/465494236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed68048c30 msgr2=0x7fed68033ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.870+0000 7fed752b2640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed68048c30 0x7fed68033ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.870+0000 7fed752b2640 1 -- 172.21.15.97:0/465494236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed68009180 msgr2=0x7fed68050ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T04:07:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.871+0000 7fed74ab1640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed68009180 0x7fed68050ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.871+0000 7fed752b2640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed68009180 0x7fed68050ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.871+0000 7fed752b2640 1 -- 172.21.15.97:0/465494236 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed680e93c0 con 0x7fed680e7d50 2024-03-29T04:07:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.871+0000 7fed74ab1640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed68009180 0x7fed68050ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:07:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.871+0000 7fed752b2640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed680e7d50 0x7fed680e8200 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fed7002d940 tx=0x7fed70063360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.872+0000 7fed6d7fa640 1 -- 172.21.15.97:0/465494236 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7008a070 con 0x7fed680e7d50 2024-03-29T04:07:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.872+0000 7fed76d15640 1 -- 172.21.15.97:0/465494236 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed680e9660 con 0x7fed680e7d50 2024-03-29T04:07:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.872+0000 7fed76d15640 1 -- 172.21.15.97:0/465494236 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fed680e9c20 con 0x7fed680e7d50 2024-03-29T04:07:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.873+0000 7fed6d7fa640 1 -- 172.21.15.97:0/465494236 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed700aee10 con 0x7fed680e7d50 2024-03-29T04:07:47.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.873+0000 7fed6d7fa640 1 -- 172.21.15.97:0/465494236 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70085620 con 0x7fed680e7d50 2024-03-29T04:07:47.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.875+0000 7fed6d7fa640 1 -- 172.21.15.97:0/465494236 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fed700af8f0 con 0x7fed680e7d50 2024-03-29T04:07:47.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.875+0000 7fed6d7fa640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fed6006c740 0x7fed6006ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.876+0000 7fed6d7fa640 1 -- 172.21.15.97:0/465494236 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fed70117910 con 0x7fed680e7d50 2024-03-29T04:07:47.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.876+0000 7fed74ab1640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fed6006c740 0x7fed6006ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.877+0000 7fed74ab1640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fed6006c740 0x7fed6006ec00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00a8b0 tx=0x7fed5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.878+0000 7fed76d15640 1 -- 172.21.15.97:0/465494236 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed50005290 con 0x7fed680e7d50 2024-03-29T04:07:47.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.883+0000 7fed6d7fa640 1 -- 172.21.15.97:0/465494236 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fed70080070 con 0x7fed680e7d50 2024-03-29T04:07:47.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.913+0000 7f953d9af640 1 Processor -- start 2024-03-29T04:07:47.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.913+0000 7f953d9af640 1 -- start start 2024-03-29T04:07:47.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.914+0000 7f953d9af640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f95381443b0 0x7f953806dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.914+0000 7f953d9af640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95380d0030 con 0x7f95380cccd0 2024-03-29T04:07:47.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.914+0000 7f953d9af640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95380d0310 con 0x7f95380cfa90 2024-03-29T04:07:47.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.914+0000 7f953d9af640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f95380b4880 con 0x7f95381443b0 2024-03-29T04:07:47.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.915+0000 7f9536ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f95381443b0 0x7f953806dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.915+0000 7f9536ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f95381443b0 0x7f953806dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55610/0 (socket says 172.21.15.97:55610) 2024-03-29T04:07:47.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.915+0000 7f9536ffd640 1 -- 172.21.15.97:0/777413694 learned_addr learned my addr 172.21.15.97:0/777413694 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.915+0000 7f9536ffd640 1 -- 172.21.15.97:0/777413694 >> v1:172.21.15.154:6789/0 conn(0x7f95380cfa90 legacy=0x7f95380b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.915+0000 7f9536ffd640 1 -- 172.21.15.97:0/777413694 >> v1:172.21.15.97:6789/0 conn(0x7f95380cccd0 legacy=0x7f95380cd0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:07:47.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.915+0000 7f227cdef640 1 Processor -- start 2024-03-29T04:07:47.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.915+0000 7f227cdef640 1 -- start start 2024-03-29T04:07:47.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.915+0000 7fdac8e1c640 1 Processor -- start 2024-03-29T04:07:47.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.915+0000 7fdac8e1c640 1 -- start start 2024-03-29T04:07:47.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.915+0000 7f9536ffd640 1 -- 172.21.15.97:0/777413694 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95380b4b60 con 0x7f95381443b0 2024-03-29T04:07:47.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f9536ffd640 1 --2- 172.21.15.97:0/777413694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95381443b0 0x7f953806dfe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9530009d40 tx=0x7f953002d070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=194b2d90480c78de server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f227cdef640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f227806da60 0x7f22780c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f227cdef640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22780c66d0 0x7f227815abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f227cdef640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f227815b2f0 con 0x7f227806de40 2024-03-29T04:07:47.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f227cdef640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f22780cf300 con 0x7f227806da60 2024-03-29T04:07:47.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f227cdef640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f227815b5d0 con 0x7f22780c66d0 2024-03-29T04:07:47.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f9535ffb640 1 -- 172.21.15.97:0/777413694 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953002dc10 con 0x7f95381443b0 2024-03-29T04:07:47.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f1471831640 1 Processor -- start 2024-03-29T04:07:47.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f2275d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22780c66d0 0x7f227815abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f2275d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22780c66d0 0x7f227815abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:35854/0 (socket says 172.21.15.97:35854) 2024-03-29T04:07:47.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f2276d9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f227806de40 0x7f227815e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:40476/0 (socket says 172.21.15.97:40476) 2024-03-29T04:07:47.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f9535ffb640 1 -- 172.21.15.97:0/777413694 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953002ddb0 con 0x7f95381443b0 2024-03-29T04:07:47.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f2276d9d640 1 -- 172.21.15.97:0/391004440 learned_addr learned my addr 172.21.15.97:0/391004440 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:47.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f227659c640 1 --2- 172.21.15.97:0/391004440 >> v2:172.21.15.154:3300/0 conn(0x7f227806da60 0x7f22780c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7fdac8e1c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdabc048030 0x7fdabc04fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7fdac8e1c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdabc037ba0 0x7fdabc038000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7fdac8e1c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdabc038540 0x7fdabc0389d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7fdac8e1c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fdabc048410 con 0x7fdabc037ba0 2024-03-29T04:07:47.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7fdac8e1c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fdabc054790 con 0x7fdabc038540 2024-03-29T04:07:47.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7fdac8e1c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdabc0536f0 con 0x7fdabc048030 2024-03-29T04:07:47.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7fdac3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdabc038540 0x7fdabc0389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7fdac2ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdabc037ba0 0x7fdabc038000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7fdac3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdabc038540 0x7fdabc0389d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55616/0 (socket says 172.21.15.97:55616) 2024-03-29T04:07:47.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7fdac3fff640 1 -- 172.21.15.97:0/3959791043 learned_addr learned my addr 172.21.15.97:0/3959791043 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7fdac3fff640 1 -- 172.21.15.97:0/3959791043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdabc048030 msgr2=0x7fdabc04fa80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:07:47.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7fdac3fff640 1 --2- 172.21.15.97:0/3959791043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdabc048030 0x7fdabc04fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7fdac3fff640 1 -- 172.21.15.97:0/3959791043 >> v2:172.21.15.97:3300/0 conn(0x7fdabc037ba0 msgr2=0x7fdabc038000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:47.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f09ed972640 1 Processor -- start 2024-03-29T04:07:47.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7fdac3fff640 1 --2- 172.21.15.97:0/3959791043 >> v2:172.21.15.97:3300/0 conn(0x7fdabc037ba0 0x7fdabc038000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7fdac3fff640 1 -- 172.21.15.97:0/3959791043 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdabc0539d0 con 0x7fdabc038540 2024-03-29T04:07:47.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f2275d9b640 1 -- 172.21.15.97:0/391004440 >> v2:172.21.15.154:3300/0 conn(0x7f227806da60 msgr2=0x7f22780c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:47.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7fdac2ffd640 1 --2- 172.21.15.97:0/3959791043 >> v2:172.21.15.97:3300/0 conn(0x7fdabc037ba0 0x7fdabc038000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:47.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f2275d9b640 1 --2- 172.21.15.97:0/391004440 >> v2:172.21.15.154:3300/0 conn(0x7f227806da60 0x7f22780c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f2275d9b640 1 -- 172.21.15.97:0/391004440 >> v1:172.21.15.154:6789/0 conn(0x7f227806de40 legacy=0x7f227815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f9535ffb640 1 -- 172.21.15.97:0/777413694 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530035610 con 0x7f95381443b0 2024-03-29T04:07:47.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f2275d9b640 1 -- 172.21.15.97:0/391004440 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227815b8b0 con 0x7f22780c66d0 2024-03-29T04:07:47.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f953d9af640 1 -- 172.21.15.97:0/777413694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95381443b0 msgr2=0x7f953806dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f953d9af640 1 --2- 172.21.15.97:0/777413694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95381443b0 0x7f953806dfe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9530009d40 tx=0x7f953002d070 comp rx=0 tx=0).stop 2024-03-29T04:07:47.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f953d9af640 1 -- 172.21.15.97:0/777413694 shutdown_connections 2024-03-29T04:07:47.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f953d9af640 1 --2- 172.21.15.97:0/777413694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95381443b0 0x7f953806dfe0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f953d9af640 1 -- 172.21.15.97:0/777413694 >> 172.21.15.97:0/777413694 conn(0x7f9538072eb0 msgr2=0x7f953806b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:47.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.917+0000 7f1471831640 1 -- start start 2024-03-29T04:07:47.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f953d9af640 1 -- 172.21.15.97:0/777413694 shutdown_connections 2024-03-29T04:07:47.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f1471831640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1464006b40 0x7f1464007f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f1471831640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14640dfa50 0x7f1464006f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f1471831640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f14640342e0 0x7f1464007460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f1471831640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f14640079a0 con 0x7f14640342e0 2024-03-29T04:07:47.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f1471831640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f1464055d00 con 0x7f1464006b40 2024-03-29T04:07:47.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f1471831640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1464055fe0 con 0x7f14640dfa50 2024-03-29T04:07:47.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f953d9af640 1 -- 172.21.15.97:0/777413694 wait complete. 2024-03-29T04:07:47.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f147082f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1464006b40 0x7f1464007f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f147082f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1464006b40 0x7f1464007f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55630/0 (socket says 172.21.15.97:55630) 2024-03-29T04:07:47.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f147082f640 1 -- 172.21.15.97:0/3256382691 learned_addr learned my addr 172.21.15.97:0/3256382691 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f146bfff640 1 --2- 172.21.15.97:0/3256382691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14640dfa50 0x7f1464006f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f1471030640 1 --2- 172.21.15.97:0/3256382691 >> v2:172.21.15.97:3300/0 conn(0x7f14640342e0 0x7f1464007460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f147082f640 1 -- 172.21.15.97:0/3256382691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14640dfa50 msgr2=0x7f1464006f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:47.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f147082f640 1 --2- 172.21.15.97:0/3256382691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14640dfa50 0x7f1464006f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f147082f640 1 -- 172.21.15.97:0/3256382691 >> v2:172.21.15.97:3300/0 conn(0x7f14640342e0 msgr2=0x7f1464007460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.918+0000 7f09ed972640 1 -- start start 2024-03-29T04:07:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f09ed972640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09e8162d10 0x7f09e8166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f09ed972640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09e80b4a50 0x7f09e8166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f09ed972640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f09e81672a0 0x7f09e81630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f09ed972640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f09e80cbb30 con 0x7f09e81672a0 2024-03-29T04:07:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f09ed972640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f09e80b3b60 con 0x7f09e8162d10 2024-03-29T04:07:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f09ed972640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f09e80b3e40 con 0x7f09e80b4a50 2024-03-29T04:07:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f09ec970640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09e8162d10 0x7f09e8166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f09ec970640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09e8162d10 0x7f09e8166820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55640/0 (socket says 172.21.15.97:55640) 2024-03-29T04:07:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09e80b4a50 0x7f09e8166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f09ec970640 1 -- 172.21.15.97:0/3969991211 learned_addr learned my addr 172.21.15.97:0/3969991211 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7fdac3fff640 1 --2- 172.21.15.97:0/3959791043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdabc038540 0x7fdabc0389d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdac402d940 tx=0x7fdac40b82e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97d55b1b914dcfbc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f09ed171640 1 --2- 172.21.15.97:0/3969991211 >> v2:172.21.15.97:3300/0 conn(0x7f09e81672a0 0x7f09e81630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f5706ab7640 1 Processor -- start 2024-03-29T04:07:47.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f09ec970640 1 -- 172.21.15.97:0/3969991211 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09e80b4a50 msgr2=0x7f09e8166d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:47.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f09ec970640 1 --2- 172.21.15.97:0/3969991211 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09e80b4a50 0x7f09e8166d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f09ec970640 1 -- 172.21.15.97:0/3969991211 >> v2:172.21.15.97:3300/0 conn(0x7f09e81672a0 msgr2=0x7f09e81630f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:47.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f09ec970640 1 --2- 172.21.15.97:0/3969991211 >> v2:172.21.15.97:3300/0 conn(0x7f09e81672a0 0x7f09e81630f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f09ec970640 1 -- 172.21.15.97:0/3969991211 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e80b4120 con 0x7f09e8162d10 2024-03-29T04:07:47.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f09ed171640 1 --2- 172.21.15.97:0/3969991211 >> v2:172.21.15.97:3300/0 conn(0x7f09e81672a0 0x7f09e81630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:07:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7fdac27fc640 1 -- 172.21.15.97:0/3959791043 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40c5070 con 0x7fdabc038540 2024-03-29T04:07:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7fdac27fc640 1 -- 172.21.15.97:0/3959791043 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac40c0680 con 0x7fdabc038540 2024-03-29T04:07:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f147082f640 1 --2- 172.21.15.97:0/3256382691 >> v2:172.21.15.97:3300/0 conn(0x7f14640342e0 0x7f1464007460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.919+0000 7f147082f640 1 -- 172.21.15.97:0/3256382691 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1464036a70 con 0x7f1464006b40 2024-03-29T04:07:47.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f2275d9b640 1 --2- 172.21.15.97:0/391004440 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22780c66d0 0x7f227815abf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f22680097d0 tx=0x7f226802ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef3f83a4a8e1f00 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7fdac27fc640 1 -- 172.21.15.97:0/3959791043 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40c0af0 con 0x7fdabc038540 2024-03-29T04:07:47.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f1471030640 1 --2- 172.21.15.97:0/3256382691 >> v2:172.21.15.97:3300/0 conn(0x7f14640342e0 0x7f1464007460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:07:47.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f227559a640 1 -- 172.21.15.97:0/391004440 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268035070 con 0x7f22780c66d0 2024-03-29T04:07:47.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f227559a640 1 -- 172.21.15.97:0/391004440 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2268031850 con 0x7f22780c66d0 2024-03-29T04:07:47.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f227559a640 1 -- 172.21.15.97:0/391004440 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268031c20 con 0x7f22780c66d0 2024-03-29T04:07:47.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f953d9af640 1 Processor -- start 2024-03-29T04:07:47.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.920+0000 7f5706ab7640 1 -- start start 2024-03-29T04:07:47.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f953d9af640 1 -- start start 2024-03-29T04:07:47.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f147082f640 1 --2- 172.21.15.97:0/3256382691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1464006b40 0x7f1464007f60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1464007d80 tx=0x7f146002dea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=549031d53b30058e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f09ec970640 1 --2- 172.21.15.97:0/3969991211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09e8162d10 0x7f09e8166820 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f09e806f870 tx=0x7f09e002be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e952042c69e4f73 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7fcc7bfff640 1 Processor -- start 2024-03-29T04:07:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3959791043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdabc038540 msgr2=0x7fdabc0389d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7fdac8e1c640 1 --2- 172.21.15.97:0/3959791043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdabc038540 0x7fdabc0389d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdac402d940 tx=0x7fdac40b82e0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7fcc7bfff640 1 -- start start 2024-03-29T04:07:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f146b7fe640 1 -- 172.21.15.97:0/3256382691 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460034070 con 0x7f1464006b40 2024-03-29T04:07:47.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f146b7fe640 1 -- 172.21.15.97:0/3256382691 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f146002ec50 con 0x7f1464006b40 2024-03-29T04:07:47.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f146b7fe640 1 -- 172.21.15.97:0/3256382691 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460038400 con 0x7f1464006b40 2024-03-29T04:07:47.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3959791043 shutdown_connections 2024-03-29T04:07:47.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7fdac8e1c640 1 --2- 172.21.15.97:0/3959791043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdabc038540 0x7fdabc0389d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7fdac8e1c640 1 --2- 172.21.15.97:0/3959791043 >> v2:172.21.15.97:3300/0 conn(0x7fdabc037ba0 0x7fdabc038000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7fdac8e1c640 1 --2- 172.21.15.97:0/3959791043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdabc048030 0x7fdabc04fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3959791043 >> 172.21.15.97:0/3959791043 conn(0x7fdabc009fa0 msgr2=0x7fdabc00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:47.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3959791043 shutdown_connections 2024-03-29T04:07:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f09e77fe640 1 -- 172.21.15.97:0/3969991211 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e002c9f0 con 0x7f09e8162d10 2024-03-29T04:07:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f09e77fe640 1 -- 172.21.15.97:0/3969991211 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e002cb90 con 0x7f09e8162d10 2024-03-29T04:07:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f09e77fe640 1 -- 172.21.15.97:0/3969991211 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e00343f0 con 0x7f09e8162d10 2024-03-29T04:07:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f227cdef640 1 -- 172.21.15.97:0/391004440 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22780c66d0 msgr2=0x7f227815abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f227cdef640 1 --2- 172.21.15.97:0/391004440 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22780c66d0 0x7f227815abf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f22680097d0 tx=0x7f226802ca10 comp rx=0 tx=0).stop 2024-03-29T04:07:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f1471831640 1 -- 172.21.15.97:0/3256382691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1464006b40 msgr2=0x7f1464007f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f1471831640 1 --2- 172.21.15.97:0/3256382691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1464006b40 0x7f1464007f60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1464007d80 tx=0x7f146002dea0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f953d9af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cccd0 0x7f95380cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f1471831640 1 -- 172.21.15.97:0/3256382691 shutdown_connections 2024-03-29T04:07:47.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f227cdef640 1 -- 172.21.15.97:0/391004440 shutdown_connections 2024-03-29T04:07:47.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f227cdef640 1 --2- 172.21.15.97:0/391004440 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f22780c66d0 0x7f227815abf0 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f22680097d0 tx=0x7f226802ca10 comp rx=0 tx=0).stop 2024-03-29T04:07:47.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f227cdef640 1 --2- 172.21.15.97:0/391004440 >> v2:172.21.15.154:3300/0 conn(0x7f227806da60 0x7f22780c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f227cdef640 1 -- 172.21.15.97:0/391004440 >> 172.21.15.97:0/391004440 conn(0x7f2278072ba0 msgr2=0x7f2278072f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:47.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f1471831640 1 --2- 172.21.15.97:0/3256382691 >> v2:172.21.15.97:3300/0 conn(0x7f14640342e0 0x7f1464007460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f1471831640 1 --2- 172.21.15.97:0/3256382691 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14640dfa50 0x7f1464006f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f1471831640 1 --2- 172.21.15.97:0/3256382691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1464006b40 0x7f1464007f60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f1471831640 1 -- 172.21.15.97:0/3256382691 >> 172.21.15.97:0/3256382691 conn(0x7f146401e6b0 msgr2=0x7f14640553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:47.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7fcc7bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcc7c0ce5f0 0x7fcc7c0acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f1471831640 1 -- 172.21.15.97:0/3256382691 shutdown_connections 2024-03-29T04:07:47.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f9536ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cccd0 0x7f95380cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f9536ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cccd0 0x7f95380cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39062/0 (socket says 172.21.15.97:39062) 2024-03-29T04:07:47.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7fcc7bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc7c0ccfe0 0x7fcc7c0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7fcc7bfff640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7c15e9a0 con 0x7fcc7c15a550 2024-03-29T04:07:47.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7fcc7bfff640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7c0cd990 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7fcc7bfff640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7c06d540 con 0x7fcc7c0ce5f0 2024-03-29T04:07:47.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f9536ffd640 1 -- 172.21.15.97:0/333346295 learned_addr learned my addr 172.21.15.97:0/333346295 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f953d9af640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95380cfa90 0x7f95381f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7fcc7b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc7c0ccfe0 0x7fcc7c0cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f227cdef640 1 -- 172.21.15.97:0/391004440 shutdown_connections 2024-03-29T04:07:47.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f09ed972640 1 -- 172.21.15.97:0/3969991211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09e8162d10 msgr2=0x7f09e8166820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f09ed972640 1 --2- 172.21.15.97:0/3969991211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09e8162d10 0x7f09e8166820 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f09e806f870 tx=0x7f09e002be50 comp rx=0 tx=0).stop 2024-03-29T04:07:47.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f953d9af640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95381443b0 0x7f9538159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7fcc7b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc7c0ccfe0 0x7fcc7c0cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39068/0 (socket says 172.21.15.97:39068) 2024-03-29T04:07:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7fcc7b7fe640 1 -- 172.21.15.97:0/657361316 learned_addr learned my addr 172.21.15.97:0/657361316 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7f95367fc640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95380cfa90 0x7f95381f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7f953d9af640 1 -- 172.21.15.97:0/333346295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9538144910 con 0x7f95380cccd0 2024-03-29T04:07:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f5706ab7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f570006c520 0x7f570006c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f5706ab7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 0x7f57000b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f5706ab7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f570006e7b0 con 0x7f57001447a0 2024-03-29T04:07:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f5706ab7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f5700067a40 con 0x7f570006c520 2024-03-29T04:07:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f5706ab7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f570006ea90 con 0x7f57000b4a50 2024-03-29T04:07:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f5705054640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 0x7f57000b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f5705054640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 0x7f57000b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:35880/0 (socket says 172.21.15.97:35880) 2024-03-29T04:07:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f5705054640 1 -- 172.21.15.97:0/3536635523 learned_addr learned my addr 172.21.15.97:0/3536635523 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7f56fffff640 1 --2- 172.21.15.97:0/3536635523 >> v2:172.21.15.97:3300/0 conn(0x7f570006c520 0x7f570006c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f5705054640 1 -- 172.21.15.97:0/3536635523 >> v2:172.21.15.97:3300/0 conn(0x7f570006c520 msgr2=0x7f570006c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f5705054640 1 --2- 172.21.15.97:0/3536635523 >> v2:172.21.15.97:3300/0 conn(0x7f570006c520 0x7f570006c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f5705054640 1 -- 172.21.15.97:0/3536635523 >> v1:172.21.15.154:6789/0 conn(0x7f57001447a0 legacy=0x7f5700067330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:07:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f5705054640 1 -- 172.21.15.97:0/3536635523 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570006ed70 con 0x7f57000b4a50 2024-03-29T04:07:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7f56fffff640 1 --2- 172.21.15.97:0/3536635523 >> v2:172.21.15.97:3300/0 conn(0x7f570006c520 0x7f570006c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.921+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3959791043 wait complete. 2024-03-29T04:07:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7f95377fe640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95381443b0 0x7f9538159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7fdac8e1c640 1 Processor -- start 2024-03-29T04:07:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7f5705054640 1 --2- 172.21.15.97:0/3536635523 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 0x7f57000b4e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009870 tx=0x7f56f402b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd2e01c53a3762d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7f953d9af640 1 -- 172.21.15.97:0/333346295 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9538144790 con 0x7f95381443b0 2024-03-29T04:07:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7f953d9af640 1 -- 172.21.15.97:0/333346295 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9538067330 con 0x7f95380cfa90 2024-03-29T04:07:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7f56ff7fe640 1 -- 172.21.15.97:0/3536635523 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f402e070 con 0x7f57000b4a50 2024-03-29T04:07:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7f56ff7fe640 1 -- 172.21.15.97:0/3536635523 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f4037070 con 0x7f57000b4a50 2024-03-29T04:07:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f1471831640 1 -- 172.21.15.97:0/3256382691 wait complete. 2024-03-29T04:07:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f9536ffd640 1 -- 172.21.15.97:0/333346295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95380cfa90 msgr2=0x7f95381f0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f9536ffd640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95380cfa90 0x7f95381f0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f9536ffd640 1 -- 172.21.15.97:0/333346295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95381443b0 msgr2=0x7f9538159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f9536ffd640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95381443b0 0x7f9538159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f9536ffd640 1 -- 172.21.15.97:0/333346295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95381f12c0 con 0x7f95380cccd0 2024-03-29T04:07:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f1471831640 1 Processor -- start 2024-03-29T04:07:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f09ed972640 1 -- 172.21.15.97:0/3969991211 shutdown_connections 2024-03-29T04:07:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f227cdef640 1 -- 172.21.15.97:0/391004440 wait complete. 2024-03-29T04:07:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f1471831640 1 -- start start 2024-03-29T04:07:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f9536ffd640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cccd0 0x7f95380cacb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9530009d40 tx=0x7f9530004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f227cdef640 1 Processor -- start 2024-03-29T04:07:47.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f5706ab7640 1 -- 172.21.15.97:0/3536635523 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 msgr2=0x7f57000b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f5706ab7640 1 --2- 172.21.15.97:0/3536635523 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 0x7f57000b4e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009870 tx=0x7f56f402b880 comp rx=0 tx=0).stop 2024-03-29T04:07:47.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f227cdef640 1 -- start start 2024-03-29T04:07:47.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7fdac8e1c640 1 -- start start 2024-03-29T04:07:47.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7f951ffff640 1 -- 172.21.15.97:0/333346295 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953003f620 con 0x7f95380cccd0 2024-03-29T04:07:47.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7fdac8e1c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdabc037ba0 0x7fdabc0d8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7fdac8e1c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdabc038540 0x7fdabc0d7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7f953d9af640 1 -- 172.21.15.97:0/333346295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95381596a0 con 0x7f95380cccd0 2024-03-29T04:07:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7f953d9af640 1 -- 172.21.15.97:0/333346295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95380cb650 con 0x7f95380cccd0 2024-03-29T04:07:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7f951ffff640 1 -- 172.21.15.97:0/333346295 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9530044070 con 0x7f95380cccd0 2024-03-29T04:07:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7f951ffff640 1 -- 172.21.15.97:0/333346295 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953003fa00 con 0x7f95380cccd0 2024-03-29T04:07:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7f5706ab7640 1 -- 172.21.15.97:0/3536635523 shutdown_connections 2024-03-29T04:07:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7f5706ab7640 1 --2- 172.21.15.97:0/3536635523 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 0x7f57000b4e60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7f5706ab7640 1 --2- 172.21.15.97:0/3536635523 >> v2:172.21.15.97:3300/0 conn(0x7f570006c520 0x7f570006c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7f5706ab7640 1 -- 172.21.15.97:0/3536635523 >> 172.21.15.97:0/3536635523 conn(0x7f5700072020 msgr2=0x7f5700072430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7fcc7b7fe640 1 -- 172.21.15.97:0/657361316 >> v2:172.21.15.154:3300/0 conn(0x7fcc7c0ce5f0 msgr2=0x7fcc7c0acf20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:07:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7fcc7b7fe640 1 --2- 172.21.15.97:0/657361316 >> v2:172.21.15.154:3300/0 conn(0x7fcc7c0ce5f0 0x7fcc7c0acf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7fcc7b7fe640 1 -- 172.21.15.97:0/657361316 >> v1:172.21.15.97:6789/0 conn(0x7fcc7c15a550 legacy=0x7fcc7c15e1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:07:47.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f227cdef640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f227806da60 0x7f2278200f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f227cdef640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f227806de40 0x7f227815c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f227cdef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22780b5920 0x7f22780b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f227659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f227806da60 0x7f2278200f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f227659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f227806da60 0x7f2278200f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55676/0 (socket says 172.21.15.97:55676) 2024-03-29T04:07:47.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f1471831640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1464006b40 0x7f14640ddc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f2275d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f227806de40 0x7f227815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f1471831640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14640342e0 0x7f1464036e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f09ed972640 1 --2- 172.21.15.97:0/3969991211 >> v2:172.21.15.97:3300/0 conn(0x7f09e81672a0 0x7f09e81630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f09ed972640 1 --2- 172.21.15.97:0/3969991211 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09e80b4a50 0x7f09e8166d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f1471831640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14640dfa50 0x7f14640373a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f09ed972640 1 --2- 172.21.15.97:0/3969991211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09e8162d10 0x7f09e8166820 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.922+0000 7f09ed972640 1 -- 172.21.15.97:0/3969991211 >> 172.21.15.97:0/3969991211 conn(0x7f09e80b1cb0 msgr2=0x7f09e80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:47.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7f09ed972640 1 -- 172.21.15.97:0/3969991211 shutdown_connections 2024-03-29T04:07:47.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7f09ed972640 1 -- 172.21.15.97:0/3969991211 wait complete. 2024-03-29T04:07:47.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f09ed972640 1 Processor -- start 2024-03-29T04:07:47.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7f09ed972640 1 -- start start 2024-03-29T04:07:47.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f09ed972640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09e80b4a50 0x7f09e81621f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f09ed972640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09e8162d10 0x7f09e806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f09ed972640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09e81672a0 0x7f09e8164390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f09ed972640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8163630 con 0x7f09e81672a0 2024-03-29T04:07:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f09ed972640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8167730 con 0x7f09e8162d10 2024-03-29T04:07:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f09ed972640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f09e81637b0 con 0x7f09e80b4a50 2024-03-29T04:07:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09e8162d10 0x7f09e806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09e8162d10 0x7f09e806ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55686/0 (socket says 172.21.15.97:55686) 2024-03-29T04:07:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f09e7fff640 1 -- 172.21.15.97:0/2312160966 learned_addr learned my addr 172.21.15.97:0/2312160966 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f146bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14640342e0 0x7f1464036e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f227659c640 1 -- 172.21.15.97:0/4267092760 learned_addr learned my addr 172.21.15.97:0/4267092760 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f146bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14640342e0 0x7f1464036e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:35906/0 (socket says 172.21.15.97:35906) 2024-03-29T04:07:47.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f09ec970640 1 --2- 172.21.15.97:0/2312160966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09e80b4a50 0x7f09e81621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f146bfff640 1 -- 172.21.15.97:0/191779943 learned_addr learned my addr 172.21.15.97:0/191779943 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.926+0000 7f1471030640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14640dfa50 0x7f14640373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f5706ab7640 1 -- 172.21.15.97:0/3536635523 shutdown_connections 2024-03-29T04:07:47.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f1471831640 1 -- 172.21.15.97:0/191779943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1464036690 con 0x7f1464006b40 2024-03-29T04:07:47.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f227cdef640 1 -- 172.21.15.97:0/4267092760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f227806e2f0 con 0x7f22780b5920 2024-03-29T04:07:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f1471831640 1 -- 172.21.15.97:0/191779943 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1464034ff0 con 0x7f14640dfa50 2024-03-29T04:07:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f227cdef640 1 -- 172.21.15.97:0/4267092760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f227815be70 con 0x7f227806da60 2024-03-29T04:07:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f1471831640 1 -- 172.21.15.97:0/191779943 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1464036810 con 0x7f14640342e0 2024-03-29T04:07:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f227cdef640 1 -- 172.21.15.97:0/4267092760 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f22780c5020 con 0x7f227806de40 2024-03-29T04:07:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.923+0000 7fcc7b7fe640 1 -- 172.21.15.97:0/657361316 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c15ec80 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.924+0000 7fcc7b7fe640 1 --2- 172.21.15.97:0/657361316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc7c0ccfe0 0x7fcc7c0cd450 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c06e910 tx=0x7fcc7402bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad43f67d5cca37b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7fcc7a7fc640 1 -- 172.21.15.97:0/657361316 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7402caa0 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7fcc7a7fc640 1 -- 172.21.15.97:0/657361316 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc7402cc40 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7fcc7a7fc640 1 -- 172.21.15.97:0/657361316 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc740343f0 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fcc7bfff640 1 -- 172.21.15.97:0/657361316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc7c0ccfe0 msgr2=0x7fcc7c0cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f5706ab7640 1 -- 172.21.15.97:0/3536635523 wait complete. 2024-03-29T04:07:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fcc7bfff640 1 --2- 172.21.15.97:0/657361316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc7c0ccfe0 0x7fcc7c0cd450 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c06e910 tx=0x7fcc7402bf00 comp rx=0 tx=0).stop 2024-03-29T04:07:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fcc7bfff640 1 -- 172.21.15.97:0/657361316 shutdown_connections 2024-03-29T04:07:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fcc7bfff640 1 --2- 172.21.15.97:0/657361316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc7c0ccfe0 0x7fcc7c0cd450 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fcc7bfff640 1 --2- 172.21.15.97:0/657361316 >> v2:172.21.15.154:3300/0 conn(0x7fcc7c0ce5f0 0x7fcc7c0acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fcc7bfff640 1 -- 172.21.15.97:0/657361316 >> 172.21.15.97:0/657361316 conn(0x7fcc7c0b1c40 msgr2=0x7fcc7c0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f09e7fff640 1 -- 172.21.15.97:0/2312160966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09e80b4a50 msgr2=0x7f09e81621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f09e7fff640 1 --2- 172.21.15.97:0/2312160966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09e80b4a50 0x7f09e81621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f09e7fff640 1 -- 172.21.15.97:0/2312160966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09e81672a0 msgr2=0x7f09e8164390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f09e7fff640 1 --2- 172.21.15.97:0/2312160966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09e81672a0 0x7f09e8164390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f09e7fff640 1 -- 172.21.15.97:0/2312160966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e8165030 con 0x7f09e8162d10 2024-03-29T04:07:47.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7fdac8e1c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdabc048030 0x7fdabc04c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7fdac8e1c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc054270 con 0x7fdabc037ba0 2024-03-29T04:07:47.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7fdac8e1c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc053f90 con 0x7fdabc038540 2024-03-29T04:07:47.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.925+0000 7fdac8e1c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdabc04ffc0 con 0x7fdabc048030 2024-03-29T04:07:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f2276d9d640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22780b5920 0x7f22780b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f09e7fff640 1 --2- 172.21.15.97:0/2312160966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09e8162d10 0x7f09e806ee40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09d800e9b0 tx=0x7f09d800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f146bfff640 1 -- 172.21.15.97:0/191779943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14640dfa50 msgr2=0x7f14640373a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f227659c640 1 -- 172.21.15.97:0/4267092760 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f227806de40 msgr2=0x7f227815c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f227659c640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f227806de40 0x7f227815c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f227659c640 1 -- 172.21.15.97:0/4267092760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22780b5920 msgr2=0x7f22780b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f227659c640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22780b5920 0x7f22780b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f227659c640 1 -- 172.21.15.97:0/4267092760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2278201490 con 0x7f227806da60 2024-03-29T04:07:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fdac2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdabc038540 0x7fdabc0d7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fdac2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdabc038540 0x7fdabc0d7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55704/0 (socket says 172.21.15.97:55704) 2024-03-29T04:07:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f2275d9b640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f227806de40 0x7f227815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fdac2ffd640 1 -- 172.21.15.97:0/3825391048 learned_addr learned my addr 172.21.15.97:0/3825391048 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f2276d9d640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22780b5920 0x7f22780b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:47.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f147082f640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1464006b40 0x7f14640ddc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fdac3fff640 1 --2- 172.21.15.97:0/3825391048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdabc048030 0x7fdabc04c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f146bfff640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14640dfa50 0x7f14640373a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f09e57fa640 1 -- 172.21.15.97:0/2312160966 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8017070 con 0x7f09e8162d10 2024-03-29T04:07:47.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f09e57fa640 1 -- 172.21.15.97:0/2312160966 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d800c9a0 con 0x7f09e8162d10 2024-03-29T04:07:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f146bfff640 1 -- 172.21.15.97:0/191779943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1464006b40 msgr2=0x7f14640ddc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7f146bfff640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1464006b40 0x7f14640ddc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7f146bfff640 1 -- 172.21.15.97:0/191779943 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1464023000 con 0x7f14640342e0 2024-03-29T04:07:47.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f09e57fa640 1 -- 172.21.15.97:0/2312160966 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d800cd90 con 0x7f09e8162d10 2024-03-29T04:07:47.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fdac2ffd640 1 -- 172.21.15.97:0/3825391048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdabc048030 msgr2=0x7fdabc04c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fdac2ffd640 1 --2- 172.21.15.97:0/3825391048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdabc048030 0x7fdabc04c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fdac2ffd640 1 -- 172.21.15.97:0/3825391048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdabc037ba0 msgr2=0x7fdabc0d8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fdac2ffd640 1 --2- 172.21.15.97:0/3825391048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdabc037ba0 0x7fdabc0d8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7fdac2ffd640 1 -- 172.21.15.97:0/3825391048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdabc0d9530 con 0x7fdabc038540 2024-03-29T04:07:47.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f09ed972640 1 -- 172.21.15.97:0/2312160966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e8161710 con 0x7f09e8162d10 2024-03-29T04:07:47.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.927+0000 7f09ed972640 1 -- 172.21.15.97:0/2312160966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e8162730 con 0x7f09e8162d10 2024-03-29T04:07:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7f227659c640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f227806da60 0x7f2278200f50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f226c00d830 tx=0x7f226c00dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7fdac2ffd640 1 --2- 172.21.15.97:0/3825391048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdabc038540 0x7fdabc0d7fd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdab800ec10 tx=0x7fdab800c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7f146bfff640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14640342e0 0x7f1464036e60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f145800d800 tx=0x7f145800dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7f147082f640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1464006b40 0x7f14640ddc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:07:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7f1471030640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14640dfa50 0x7f14640373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7fdaaffff640 1 -- 172.21.15.97:0/3825391048 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8017070 con 0x7fdabc038540 2024-03-29T04:07:47.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7fcc7bfff640 1 -- 172.21.15.97:0/657361316 shutdown_connections 2024-03-29T04:07:47.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7f953d9af640 1 -- 172.21.15.97:0/333346295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9538068d50 con 0x7f95380cccd0 2024-03-29T04:07:47.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7fdaaffff640 1 -- 172.21.15.97:0/3825391048 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdab800cc30 con 0x7fdabc038540 2024-03-29T04:07:47.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3825391048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdabc0e8af0 con 0x7fdabc038540 2024-03-29T04:07:47.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3825391048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdabc0e90b0 con 0x7fdabc038540 2024-03-29T04:07:47.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.929+0000 7fdaaffff640 1 -- 172.21.15.97:0/3825391048 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8013470 con 0x7fdabc038540 2024-03-29T04:07:47.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7f14697fa640 1 -- 172.21.15.97:0/191779943 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145800b6c0 con 0x7f14640342e0 2024-03-29T04:07:47.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.929+0000 7f14697fa640 1 -- 172.21.15.97:0/191779943 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1458019070 con 0x7f14640342e0 2024-03-29T04:07:47.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.929+0000 7f14697fa640 1 -- 172.21.15.97:0/191779943 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145800ba60 con 0x7f14640342e0 2024-03-29T04:07:47.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.929+0000 7fcc7bfff640 1 -- 172.21.15.97:0/657361316 wait complete. 2024-03-29T04:07:47.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7f5706ab7640 1 Processor -- start 2024-03-29T04:07:47.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7f5706ab7640 1 -- start start 2024-03-29T04:07:47.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.929+0000 7f09ed972640 1 -- 172.21.15.97:0/2312160966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e804f8f0 con 0x7f09e8162d10 2024-03-29T04:07:47.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7f2266ffd640 1 -- 172.21.15.97:0/4267092760 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c00b6c0 con 0x7f227806da60 2024-03-29T04:07:47.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7f2266ffd640 1 -- 172.21.15.97:0/4267092760 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226c019070 con 0x7f227806da60 2024-03-29T04:07:47.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.928+0000 7f2266ffd640 1 -- 172.21.15.97:0/4267092760 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c00ba80 con 0x7f227806da60 2024-03-29T04:07:47.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.929+0000 7f227cdef640 1 -- 172.21.15.97:0/4267092760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227815f0a0 con 0x7f227806da60 2024-03-29T04:07:47.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.929+0000 7f227cdef640 1 -- 172.21.15.97:0/4267092760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2278159d10 con 0x7f227806da60 2024-03-29T04:07:47.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.930+0000 7f1471831640 1 -- 172.21.15.97:0/191779943 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14640de160 con 0x7f14640342e0 2024-03-29T04:07:47.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.930+0000 7f5706ab7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f570006c520 0x7f570015f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.930+0000 7f5706ab7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 0x7f57001f1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.930+0000 7f5706ab7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57001447a0 0x7f57000c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.930+0000 7f5706ab7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f570006f610 con 0x7f570006c520 2024-03-29T04:07:47.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.930+0000 7f5706ab7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f570006f330 con 0x7f57001447a0 2024-03-29T04:07:47.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.930+0000 7f5706ab7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f57000b53a0 con 0x7f57000b4a50 2024-03-29T04:07:47.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.930+0000 7f1471831640 1 -- 172.21.15.97:0/191779943 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1464021c40 con 0x7f14640342e0 2024-03-29T04:07:47.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.930+0000 7fcc7bfff640 1 Processor -- start 2024-03-29T04:07:47.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f56fffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 0x7f57001f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f56fffff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 0x7f57001f1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:35936/0 (socket says 172.21.15.97:35936) 2024-03-29T04:07:47.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f56fffff640 1 -- 172.21.15.97:0/3702585114 learned_addr learned my addr 172.21.15.97:0/3702585114 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f5705054640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57001447a0 0x7f57000c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f5704853640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f570006c520 0x7f570015f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f09e57fa640 1 -- 172.21.15.97:0/2312160966 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f09d801a070 con 0x7f09e8162d10 2024-03-29T04:07:47.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f56fffff640 1 -- 172.21.15.97:0/3702585114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57001447a0 msgr2=0x7f57000c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f56fffff640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57001447a0 0x7f57000c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f56fffff640 1 -- 172.21.15.97:0/3702585114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f570006c520 msgr2=0x7f570015f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f56fffff640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f570006c520 0x7f570015f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f56fffff640 1 -- 172.21.15.97:0/3702585114 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5700158e40 con 0x7f57000b4a50 2024-03-29T04:07:47.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.930+0000 7fcc7bfff640 1 -- start start 2024-03-29T04:07:47.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f09e57fa640 1 --2- 172.21.15.97:0/2312160966 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f09d406c630 0x7f09d406eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7f56fffff640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 0x7f57001f1290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f56f8007e40 tx=0x7f56f800d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fcc7bfff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc7c0ccfe0 0x7fcc7c151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7f09e57fa640 1 -- 172.21.15.97:0/2312160966 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f09d808b9f0 con 0x7f09e8162d10 2024-03-29T04:07:47.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3825391048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa4005290 con 0x7fdabc038540 2024-03-29T04:07:47.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fcc7bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc7c15a550 0x7fcc7c15b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fdaaffff640 1 -- 172.21.15.97:0/3825391048 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdab801a070 con 0x7fdabc038540 2024-03-29T04:07:47.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7f56fd7fa640 1 -- 172.21.15.97:0/3702585114 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f801a070 con 0x7f57000b4a50 2024-03-29T04:07:47.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7f56fd7fa640 1 -- 172.21.15.97:0/3702585114 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f8003b90 con 0x7f57000b4a50 2024-03-29T04:07:47.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7f56fd7fa640 1 -- 172.21.15.97:0/3702585114 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f800de80 con 0x7f57000b4a50 2024-03-29T04:07:47.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fcc7affd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc7c0ccfe0 0x7fcc7c151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fcc7affd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc7c0ccfe0 0x7fcc7c151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:35938/0 (socket says 172.21.15.97:35938) 2024-03-29T04:07:47.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fcc7bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc7c15b880 0x7fcc7c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f2266ffd640 1 -- 172.21.15.97:0/4267092760 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f226c01e460 con 0x7f227806da60 2024-03-29T04:07:47.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7f2266ffd640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f225806c560 0x7f225806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7f2266ffd640 1 -- 172.21.15.97:0/4267092760 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f226c08bd50 con 0x7f227806da60 2024-03-29T04:07:47.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fcc7affd640 1 -- 172.21.15.97:0/1821509958 learned_addr learned my addr 172.21.15.97:0/1821509958 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:47.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fcc7b7fe640 1 --2- 172.21.15.97:0/1821509958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc7c15b880 0x7fcc7c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fcc7bfff640 1 -- 172.21.15.97:0/1821509958 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c06d790 con 0x7fcc7c15a550 2024-03-29T04:07:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fcc7bfff640 1 -- 172.21.15.97:0/1821509958 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c0ce9f0 con 0x7fcc7c15b880 2024-03-29T04:07:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fcc7bfff640 1 -- 172.21.15.97:0/1821509958 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7c0ad460 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7f5706ab7640 1 -- 172.21.15.97:0/3702585114 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57001f7880 con 0x7f57000b4a50 2024-03-29T04:07:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7f5706ab7640 1 -- 172.21.15.97:0/3702585114 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f570015d650 con 0x7f57000b4a50 2024-03-29T04:07:47.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.933+0000 7f2275d9b640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f225806c560 0x7f225806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.933+0000 7f09ec970640 1 --2- 172.21.15.97:0/2312160966 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f09d406c630 0x7f09d406eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fdaaffff640 1 --2- 172.21.15.97:0/3825391048 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdaa806c600 0x7fdaa806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.932+0000 7fdaaffff640 1 -- 172.21.15.97:0/3825391048 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdab808b520 con 0x7fdabc038540 2024-03-29T04:07:47.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.933+0000 7fdac37fe640 1 --2- 172.21.15.97:0/3825391048 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdaa806c600 0x7fdaa806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.929+0000 7f951ffff640 1 -- 172.21.15.97:0/333346295 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f953003fba0 con 0x7f95380cccd0 2024-03-29T04:07:47.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.930+0000 7f951ffff640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f952406c530 0x7f952406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.930+0000 7f951ffff640 1 -- 172.21.15.97:0/333346295 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f95300ade70 con 0x7f95380cccd0 2024-03-29T04:07:47.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.930+0000 7f95367fc640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f952406c530 0x7f952406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.931+0000 7f95367fc640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f952406c530 0x7f952406e9f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f953806c370 tx=0x7f9528012040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.933+0000 7fcc7affd640 1 -- 172.21.15.97:0/1821509958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc7c15b880 msgr2=0x7fcc7c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.933+0000 7fcc7affd640 1 --2- 172.21.15.97:0/1821509958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc7c15b880 0x7fcc7c0cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.933+0000 7fcc7affd640 1 -- 172.21.15.97:0/1821509958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc7c15a550 msgr2=0x7fcc7c15b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:47.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.933+0000 7fcc7affd640 1 --2- 172.21.15.97:0/1821509958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc7c15a550 0x7fcc7c15b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:47.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.933+0000 7fcc7affd640 1 -- 172.21.15.97:0/1821509958 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c15beb0 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.933+0000 7f2275d9b640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f225806c560 0x7f225806ea20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2268037000 tx=0x7f2268008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.933+0000 7f09ec970640 1 --2- 172.21.15.97:0/2312160966 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f09d406c630 0x7f09d406eaf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f09e0035000 tx=0x7f09e0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.934+0000 7fdac37fe640 1 --2- 172.21.15.97:0/3825391048 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdaa806c600 0x7fdaa806eac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdab00014d0 tx=0x7fdab0007a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.933+0000 7f5706ab7640 1 -- 172.21.15.97:0/3702585114 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5700075060 con 0x7f57000b4a50 2024-03-29T04:07:47.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.934+0000 7fcc7affd640 1 --2- 172.21.15.97:0/1821509958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc7c0ccfe0 0x7fcc7c151810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00b450 tx=0x7fcc6c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.934+0000 7fcc737fe640 1 -- 172.21.15.97:0/1821509958 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c015020 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.934+0000 7fcc737fe640 1 -- 172.21.15.97:0/1821509958 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c0118b0 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.934+0000 7fcc737fe640 1 -- 172.21.15.97:0/1821509958 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c011ca0 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.934+0000 7fcc7bfff640 1 -- 172.21.15.97:0/1821509958 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7c15c190 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.934+0000 7fcc7bfff640 1 -- 172.21.15.97:0/1821509958 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7c0c8dc0 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.934+0000 7f227cdef640 1 -- 172.21.15.97:0/4267092760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2278068640 con 0x7f227806da60 2024-03-29T04:07:47.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.935+0000 7f56fd7fa640 1 -- 172.21.15.97:0/3702585114 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f56f801d070 con 0x7f57000b4a50 2024-03-29T04:07:47.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.935+0000 7f1471831640 1 -- 172.21.15.97:0/191779943 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1464005380 con 0x7f14640342e0 2024-03-29T04:07:47.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.935+0000 7fcc7bfff640 1 -- 172.21.15.97:0/1821509958 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc7c04f8f0 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.935+0000 7f56fd7fa640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f56ec06c530 0x7f56ec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.935+0000 7f14697fa640 1 -- 172.21.15.97:0/191779943 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f145800bc00 con 0x7f14640342e0 2024-03-29T04:07:47.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.936+0000 7f56fd7fa640 1 -- 172.21.15.97:0/3702585114 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f56f808b9b0 con 0x7f57000b4a50 2024-03-29T04:07:47.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.936+0000 7f5704853640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f56ec06c530 0x7f56ec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.936+0000 7f09e57fa640 1 -- 172.21.15.97:0/2312160966 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f09d805a080 con 0x7f09e8162d10 2024-03-29T04:07:47.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.936+0000 7f14697fa640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f145006c580 0x7f145006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.936+0000 7f14697fa640 1 -- 172.21.15.97:0/191779943 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f145808b980 con 0x7f14640342e0 2024-03-29T04:07:47.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.936+0000 7f147082f640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f145006c580 0x7f145006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.936+0000 7f5704853640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f56ec06c530 0x7f56ec06e9f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56f0006fd0 tx=0x7f56f00090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.936+0000 7f147082f640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f145006c580 0x7f145006ea40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1460039000 tx=0x7f1460034000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.938+0000 7f56fd7fa640 1 -- 172.21.15.97:0/3702585114 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5700075060 con 0x7f57000b4a50 2024-03-29T04:07:47.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.939+0000 7f951ffff640 1 -- 172.21.15.97:0/333346295 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f953007bda0 con 0x7f95380cccd0 2024-03-29T04:07:47.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.939+0000 7fcc737fe640 1 -- 172.21.15.97:0/1821509958 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcc6c018020 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.940+0000 7fcc737fe640 1 --2- 172.21.15.97:0/1821509958 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcc5c06c580 0x7fcc5c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:47.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.940+0000 7fcc73fff640 1 --2- 172.21.15.97:0/1821509958 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcc5c06c580 0x7fcc5c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.940+0000 7fdaaffff640 1 -- 172.21.15.97:0/3825391048 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdab80594d0 con 0x7fdabc038540 2024-03-29T04:07:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.940+0000 7fcc73fff640 1 --2- 172.21.15.97:0/1821509958 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcc5c06c580 0x7fcc5c06ea40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c151e10 tx=0x7fcc64006c80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.943+0000 7fcc737fe640 1 -- 172.21.15.97:0/1821509958 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcc6c057e60 con 0x7fcc7c0ccfe0 2024-03-29T04:07:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.943+0000 7f14697fa640 1 -- 172.21.15.97:0/191779943 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f145805a080 con 0x7f14640342e0 2024-03-29T04:07:47.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.945+0000 7f2266ffd640 1 -- 172.21.15.97:0/4267092760 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f226c05a360 con 0x7f227806da60 2024-03-29T04:07:47.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:47.948+0000 7fcc737fe640 1 -- 172.21.15.97:0/1821509958 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcc6c01f020 con 0x7fcc7c0ccfe0 2024-03-29T04:07:48.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.288+0000 7fed76d15640 1 -- 172.21.15.97:0/465494236 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fed50005580 con 0x7fed680e7d50 2024-03-29T04:07:48.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.289+0000 7fed6d7fa640 1 -- 172.21.15.97:0/465494236 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fed700e59e0 con 0x7fed680e7d50 2024-03-29T04:07:48.269 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2024-03-29T04:07:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.292+0000 7fed5affd640 1 -- 172.21.15.97:0/465494236 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fed6006c740 msgr2=0x7fed6006ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.292+0000 7fed5affd640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fed6006c740 0x7fed6006ec00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00a8b0 tx=0x7fed5c008040 comp rx=0 tx=0).stop 2024-03-29T04:07:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.292+0000 7fed5affd640 1 -- 172.21.15.97:0/465494236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed680e7d50 msgr2=0x7fed680e8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.292+0000 7fed5affd640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed680e7d50 0x7fed680e8200 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fed7002d940 tx=0x7fed70063360 comp rx=0 tx=0).stop 2024-03-29T04:07:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.293+0000 7fed5affd640 1 -- 172.21.15.97:0/465494236 shutdown_connections 2024-03-29T04:07:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.293+0000 7fed5affd640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fed6006c740 0x7fed6006ec00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.293+0000 7fed5affd640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed680e7d50 0x7fed680e8200 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.293+0000 7fed5affd640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed68048c30 0x7fed68033ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.293+0000 7fed5affd640 1 --2- 172.21.15.97:0/465494236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed68009180 0x7fed68050ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.293+0000 7fed5affd640 1 -- 172.21.15.97:0/465494236 >> 172.21.15.97:0/465494236 conn(0x7fed6801ef30 msgr2=0x7fed68054b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:48.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.293+0000 7fed5affd640 1 -- 172.21.15.97:0/465494236 shutdown_connections 2024-03-29T04:07:48.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.293+0000 7fed5affd640 1 -- 172.21.15.97:0/465494236 wait complete. 2024-03-29T04:07:48.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-03-29T04:07:48.290 DEBUG:teuthology.parallel:result is None 2024-03-29T04:07:48.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.330+0000 7f227cdef640 1 -- 172.21.15.97:0/4267092760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f22780d0ef0 con 0x7f227806da60 2024-03-29T04:07:48.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.330+0000 7f2266ffd640 1 -- 172.21.15.97:0/4267092760 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f226c059c60 con 0x7f227806da60 2024-03-29T04:07:48.310 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2024-03-29T04:07:48.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.332+0000 7f227cdef640 1 -- 172.21.15.97:0/4267092760 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f225806c560 msgr2=0x7f225806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.332+0000 7f227cdef640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f225806c560 0x7f225806ea20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2268037000 tx=0x7f2268008030 comp rx=0 tx=0).stop 2024-03-29T04:07:48.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.332+0000 7f227cdef640 1 -- 172.21.15.97:0/4267092760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f227806da60 msgr2=0x7f2278200f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.332+0000 7f227cdef640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f227806da60 0x7f2278200f50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f226c00d830 tx=0x7f226c00dcf0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.332+0000 7f227cdef640 1 -- 172.21.15.97:0/4267092760 shutdown_connections 2024-03-29T04:07:48.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.332+0000 7f227cdef640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f225806c560 0x7f225806ea20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.332+0000 7f227cdef640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22780b5920 0x7f22780b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.332+0000 7f227cdef640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f227806de40 0x7f227815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.332+0000 7f227cdef640 1 --2- 172.21.15.97:0/4267092760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f227806da60 0x7f2278200f50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.332+0000 7f227cdef640 1 -- 172.21.15.97:0/4267092760 >> 172.21.15.97:0/4267092760 conn(0x7f2278072ba0 msgr2=0x7f2278157770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:48.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.332+0000 7f227cdef640 1 -- 172.21.15.97:0/4267092760 shutdown_connections 2024-03-29T04:07:48.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.333+0000 7f227cdef640 1 -- 172.21.15.97:0/4267092760 wait complete. 2024-03-29T04:07:48.323 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-03-29T04:07:48.323 DEBUG:teuthology.parallel:result is None 2024-03-29T04:07:48.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.349+0000 7f1471831640 1 -- 172.21.15.97:0/191779943 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1464054950 con 0x7f14640342e0 2024-03-29T04:07:48.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.349+0000 7f14697fa640 1 -- 172.21.15.97:0/191779943 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1458059980 con 0x7f14640342e0 2024-03-29T04:07:48.329 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2024-03-29T04:07:48.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.352+0000 7f1471831640 1 -- 172.21.15.97:0/191779943 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f145006c580 msgr2=0x7f145006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.352+0000 7f1471831640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f145006c580 0x7f145006ea40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1460039000 tx=0x7f1460034000 comp rx=0 tx=0).stop 2024-03-29T04:07:48.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.352+0000 7f1471831640 1 -- 172.21.15.97:0/191779943 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14640342e0 msgr2=0x7f1464036e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.352+0000 7f1471831640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14640342e0 0x7f1464036e60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f145800d800 tx=0x7f145800dcc0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.352+0000 7f1471831640 1 -- 172.21.15.97:0/191779943 shutdown_connections 2024-03-29T04:07:48.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.352+0000 7f1471831640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f145006c580 0x7f145006ea40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.352+0000 7f1471831640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14640dfa50 0x7f14640373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.352+0000 7f1471831640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14640342e0 0x7f1464036e60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.352+0000 7f1471831640 1 --2- 172.21.15.97:0/191779943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1464006b40 0x7f14640ddc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.352+0000 7f1471831640 1 -- 172.21.15.97:0/191779943 >> 172.21.15.97:0/191779943 conn(0x7f146401e6b0 msgr2=0x7f146400b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:48.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.353+0000 7f1471831640 1 -- 172.21.15.97:0/191779943 shutdown_connections 2024-03-29T04:07:48.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.353+0000 7f1471831640 1 -- 172.21.15.97:0/191779943 wait complete. 2024-03-29T04:07:48.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.358+0000 7f09ed972640 1 -- 172.21.15.97:0/2312160966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f09e80cd720 con 0x7f09e8162d10 2024-03-29T04:07:48.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.358+0000 7f09e57fa640 1 -- 172.21.15.97:0/2312160966 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f09d8059980 con 0x7f09e8162d10 2024-03-29T04:07:48.338 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2024-03-29T04:07:48.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.359+0000 7f953d9af640 1 -- 172.21.15.97:0/333346295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f95380b3f70 con 0x7f95380cccd0 2024-03-29T04:07:48.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.360+0000 7f951ffff640 1 -- 172.21.15.97:0/333346295 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f95300800a0 con 0x7f95380cccd0 2024-03-29T04:07:48.340 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2024-03-29T04:07:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.361+0000 7f09ed972640 1 -- 172.21.15.97:0/2312160966 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f09d406c630 msgr2=0x7f09d406eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.361+0000 7f09ed972640 1 --2- 172.21.15.97:0/2312160966 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f09d406c630 0x7f09d406eaf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f09e0035000 tx=0x7f09e0030000 comp rx=0 tx=0).stop 2024-03-29T04:07:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.361+0000 7f09ed972640 1 -- 172.21.15.97:0/2312160966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09e8162d10 msgr2=0x7f09e806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.361+0000 7f09ed972640 1 --2- 172.21.15.97:0/2312160966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09e8162d10 0x7f09e806ee40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09d800e9b0 tx=0x7f09d800ee70 comp rx=0 tx=0).stop 2024-03-29T04:07:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f09ed972640 1 -- 172.21.15.97:0/2312160966 shutdown_connections 2024-03-29T04:07:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f09ed972640 1 --2- 172.21.15.97:0/2312160966 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f09d406c630 0x7f09d406eaf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f09ed972640 1 --2- 172.21.15.97:0/2312160966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09e81672a0 0x7f09e8164390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f09ed972640 1 --2- 172.21.15.97:0/2312160966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09e8162d10 0x7f09e806ee40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f953d9af640 1 -- 172.21.15.97:0/333346295 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f952406c530 msgr2=0x7f952406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f953d9af640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f952406c530 0x7f952406e9f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f953806c370 tx=0x7f9528012040 comp rx=0 tx=0).stop 2024-03-29T04:07:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f953d9af640 1 -- 172.21.15.97:0/333346295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cccd0 msgr2=0x7f95380cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f953d9af640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cccd0 0x7f95380cacb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9530009d40 tx=0x7f9530004a40 comp rx=0 tx=0).stop 2024-03-29T04:07:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f953d9af640 1 -- 172.21.15.97:0/333346295 shutdown_connections 2024-03-29T04:07:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f953d9af640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f952406c530 0x7f952406e9f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f953d9af640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95381443b0 0x7f9538159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f953d9af640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f95380cfa90 0x7f95381f0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f953d9af640 1 --2- 172.21.15.97:0/333346295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95380cccd0 0x7f95380cacb0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9530009d40 tx=0x7f9530004a40 comp rx=0 tx=0).stop 2024-03-29T04:07:48.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f953d9af640 1 -- 172.21.15.97:0/333346295 >> 172.21.15.97:0/333346295 conn(0x7f9538072eb0 msgr2=0x7f953806c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f09ed972640 1 --2- 172.21.15.97:0/2312160966 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09e80b4a50 0x7f09e81621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f09ed972640 1 -- 172.21.15.97:0/2312160966 >> 172.21.15.97:0/2312160966 conn(0x7f09e80b1cb0 msgr2=0x7f09e8072040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f09ed972640 1 -- 172.21.15.97:0/2312160966 shutdown_connections 2024-03-29T04:07:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f09ed972640 1 -- 172.21.15.97:0/2312160966 wait complete. 2024-03-29T04:07:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f953d9af640 1 -- 172.21.15.97:0/333346295 shutdown_connections 2024-03-29T04:07:48.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7f953d9af640 1 -- 172.21.15.97:0/333346295 wait complete. 2024-03-29T04:07:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.362+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3825391048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fdaa4005580 con 0x7fdabc038540 2024-03-29T04:07:48.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.363+0000 7fdaaffff640 1 -- 172.21.15.97:0/3825391048 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdab805d7d0 con 0x7fdabc038540 2024-03-29T04:07:48.345 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2024-03-29T04:07:48.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-03-29T04:07:48.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.366+0000 7f5706ab7640 1 -- 172.21.15.97:0/3702585114 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5700068d50 con 0x7f57000b4a50 2024-03-29T04:07:48.347 DEBUG:teuthology.parallel:result is None 2024-03-29T04:07:48.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.366+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3825391048 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdaa806c600 msgr2=0x7fdaa806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.366+0000 7fdac8e1c640 1 --2- 172.21.15.97:0/3825391048 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdaa806c600 0x7fdaa806eac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdab00014d0 tx=0x7fdab0007a50 comp rx=0 tx=0).stop 2024-03-29T04:07:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.366+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3825391048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdabc038540 msgr2=0x7fdabc0d7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.366+0000 7fdac8e1c640 1 --2- 172.21.15.97:0/3825391048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdabc038540 0x7fdabc0d7fd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdab800ec10 tx=0x7fdab800c520 comp rx=0 tx=0).stop 2024-03-29T04:07:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.366+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3825391048 shutdown_connections 2024-03-29T04:07:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.366+0000 7fdac8e1c640 1 --2- 172.21.15.97:0/3825391048 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdaa806c600 0x7fdaa806eac0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.366+0000 7fdac8e1c640 1 --2- 172.21.15.97:0/3825391048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdabc048030 0x7fdabc04c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.366+0000 7fdac8e1c640 1 --2- 172.21.15.97:0/3825391048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdabc038540 0x7fdabc0d7fd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.366+0000 7fdac8e1c640 1 --2- 172.21.15.97:0/3825391048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdabc037ba0 0x7fdabc0d8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.366+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3825391048 >> 172.21.15.97:0/3825391048 conn(0x7fdabc009fa0 msgr2=0x7fdabc0058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:48.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.366+0000 7f56fd7fa640 1 -- 172.21.15.97:0/3702585114 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f56f8010070 con 0x7f57000b4a50 2024-03-29T04:07:48.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.367+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3825391048 shutdown_connections 2024-03-29T04:07:48.349 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2024-03-29T04:07:48.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.367+0000 7fdac8e1c640 1 -- 172.21.15.97:0/3825391048 wait complete. 2024-03-29T04:07:48.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.370+0000 7f5706ab7640 1 -- 172.21.15.97:0/3702585114 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f56ec06c530 msgr2=0x7f56ec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.370+0000 7f5706ab7640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f56ec06c530 0x7f56ec06e9f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56f0006fd0 tx=0x7f56f00090e0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.370+0000 7f5706ab7640 1 -- 172.21.15.97:0/3702585114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 msgr2=0x7f57001f1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.370+0000 7f5706ab7640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 0x7f57001f1290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f56f8007e40 tx=0x7f56f800d4a0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.370+0000 7f5706ab7640 1 -- 172.21.15.97:0/3702585114 shutdown_connections 2024-03-29T04:07:48.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.370+0000 7f5706ab7640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f56ec06c530 0x7f56ec06e9f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.370+0000 7f5706ab7640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57001447a0 0x7f57000c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.370+0000 7f5706ab7640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57000b4a50 0x7f57001f1290 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.370+0000 7f5706ab7640 1 --2- 172.21.15.97:0/3702585114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f570006c520 0x7f570015f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.370+0000 7f5706ab7640 1 -- 172.21.15.97:0/3702585114 >> 172.21.15.97:0/3702585114 conn(0x7f5700072020 msgr2=0x7f570015a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:48.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.370+0000 7f5706ab7640 1 -- 172.21.15.97:0/3702585114 shutdown_connections 2024-03-29T04:07:48.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.370+0000 7f5706ab7640 1 -- 172.21.15.97:0/3702585114 wait complete. 2024-03-29T04:07:48.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.374+0000 7fcc7bfff640 1 -- 172.21.15.97:0/1821509958 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fcc7c06f210 con 0x7fcc7c0ccfe0 2024-03-29T04:07:48.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.374+0000 7fcc737fe640 1 -- 172.21.15.97:0/1821509958 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcc6c00d040 con 0x7fcc7c0ccfe0 2024-03-29T04:07:48.354 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2024-03-29T04:07:48.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-03-29T04:07:48.357 DEBUG:teuthology.parallel:result is None 2024-03-29T04:07:48.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.377+0000 7fcc7bfff640 1 -- 172.21.15.97:0/1821509958 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcc5c06c580 msgr2=0x7fcc5c06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.377+0000 7fcc7bfff640 1 --2- 172.21.15.97:0/1821509958 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcc5c06c580 0x7fcc5c06ea40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c151e10 tx=0x7fcc64006c80 comp rx=0 tx=0).stop 2024-03-29T04:07:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.377+0000 7fcc7bfff640 1 -- 172.21.15.97:0/1821509958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc7c0ccfe0 msgr2=0x7fcc7c151810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.377+0000 7fcc7bfff640 1 --2- 172.21.15.97:0/1821509958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc7c0ccfe0 0x7fcc7c151810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00b450 tx=0x7fcc6c00b910 comp rx=0 tx=0).stop 2024-03-29T04:07:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.377+0000 7fcc7bfff640 1 -- 172.21.15.97:0/1821509958 shutdown_connections 2024-03-29T04:07:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.377+0000 7fcc7bfff640 1 --2- 172.21.15.97:0/1821509958 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcc5c06c580 0x7fcc5c06ea40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.377+0000 7fcc7bfff640 1 --2- 172.21.15.97:0/1821509958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc7c15b880 0x7fcc7c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.377+0000 7fcc7bfff640 1 --2- 172.21.15.97:0/1821509958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc7c15a550 0x7fcc7c15b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.377+0000 7fcc7bfff640 1 --2- 172.21.15.97:0/1821509958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc7c0ccfe0 0x7fcc7c151810 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.377+0000 7fcc7bfff640 1 -- 172.21.15.97:0/1821509958 >> 172.21.15.97:0/1821509958 conn(0x7fcc7c0b1c40 msgr2=0x7fcc7c159900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.377+0000 7fcc7bfff640 1 -- 172.21.15.97:0/1821509958 shutdown_connections 2024-03-29T04:07:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.377+0000 7fcc7bfff640 1 -- 172.21.15.97:0/1821509958 wait complete. 2024-03-29T04:07:48.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-03-29T04:07:48.359 DEBUG:teuthology.parallel:result is None 2024-03-29T04:07:48.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-03-29T04:07:48.361 DEBUG:teuthology.parallel:result is None 2024-03-29T04:07:48.367 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-03-29T04:07:48.367 DEBUG:teuthology.parallel:result is None 2024-03-29T04:07:48.368 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-03-29T04:07:48.369 DEBUG:teuthology.parallel:result is None 2024-03-29T04:07:48.369 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-29T04:07:48.369 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:07:48.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.530+0000 7fdfd393d640 1 Processor -- start 2024-03-29T04:07:48.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.530+0000 7fdfd393d640 1 -- start start 2024-03-29T04:07:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.531+0000 7fdfd393d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdfcc15e6e0 0x7fdfcc09b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.531+0000 7fdfd393d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc162ab0 0x7fdfcc0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.531+0000 7fdfd393d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfcc0d1a10 con 0x7fdfcc0d0660 2024-03-29T04:07:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.531+0000 7fdfd393d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fdfcc0d0e90 con 0x7fdfcc15e6e0 2024-03-29T04:07:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.531+0000 7fdfd393d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc0d1170 con 0x7fdfcc162ab0 2024-03-29T04:07:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.531+0000 7fdfd16d9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdfcc15e6e0 0x7fdfcc09b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.531+0000 7fdfd16d9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdfcc15e6e0 0x7fdfcc09b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39140/0 (socket says 172.21.15.97:39140) 2024-03-29T04:07:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.531+0000 7fdfd16d9640 1 -- 172.21.15.97:0/2892160526 learned_addr learned my addr 172.21.15.97:0/2892160526 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.531+0000 7fdfd1eda640 1 --2- 172.21.15.97:0/2892160526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc162ab0 0x7fdfcc0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:48.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.531+0000 7fdfd16d9640 1 -- 172.21.15.97:0/2892160526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc162ab0 msgr2=0x7fdfcc0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:48.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.531+0000 7fdfd16d9640 1 --2- 172.21.15.97:0/2892160526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc162ab0 0x7fdfcc0b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.531+0000 7fdfd16d9640 1 -- 172.21.15.97:0/2892160526 >> v1:172.21.15.154:6789/0 conn(0x7fdfcc0d0660 legacy=0x7fdfcc1623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.531+0000 7fdfd16d9640 1 -- 172.21.15.97:0/2892160526 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc0d1450 con 0x7fdfcc15e6e0 2024-03-29T04:07:48.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.532+0000 7fdfd16d9640 1 --2- 172.21.15.97:0/2892160526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc15e6e0 0x7fdfcc09b6e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009e50 tx=0x7fdfc802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd57a7c296536d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:48.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.532+0000 7fdfc3fff640 1 -- 172.21.15.97:0/2892160526 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc802cb00 con 0x7fdfcc15e6e0 2024-03-29T04:07:48.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.532+0000 7fdfc3fff640 1 -- 172.21.15.97:0/2892160526 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc802cca0 con 0x7fdfcc15e6e0 2024-03-29T04:07:48.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.532+0000 7fdfc3fff640 1 -- 172.21.15.97:0/2892160526 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8034610 con 0x7fdfcc15e6e0 2024-03-29T04:07:48.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.533+0000 7fdfd393d640 1 -- 172.21.15.97:0/2892160526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc15e6e0 msgr2=0x7fdfcc09b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.533+0000 7fdfd393d640 1 --2- 172.21.15.97:0/2892160526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc15e6e0 0x7fdfcc09b6e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009e50 tx=0x7fdfc802bf60 comp rx=0 tx=0).stop 2024-03-29T04:07:48.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.533+0000 7fdfd393d640 1 -- 172.21.15.97:0/2892160526 shutdown_connections 2024-03-29T04:07:48.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.533+0000 7fdfd393d640 1 --2- 172.21.15.97:0/2892160526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc162ab0 0x7fdfcc0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.533+0000 7fdfd393d640 1 --2- 172.21.15.97:0/2892160526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc15e6e0 0x7fdfcc09b6e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.533+0000 7fdfd393d640 1 -- 172.21.15.97:0/2892160526 >> 172.21.15.97:0/2892160526 conn(0x7fdfcc0b3f10 msgr2=0x7fdfcc0b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:48.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.533+0000 7fdfd393d640 1 -- 172.21.15.97:0/2892160526 shutdown_connections 2024-03-29T04:07:48.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.533+0000 7fdfd393d640 1 -- 172.21.15.97:0/2892160526 wait complete. 2024-03-29T04:07:48.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.534+0000 7fdfd393d640 1 Processor -- start 2024-03-29T04:07:48.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.534+0000 7fdfd393d640 1 -- start start 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd393d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc0d0660 0x7fdfcc0c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd393d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc15e6e0 0x7fdfcc159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd393d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc162ab0 0x7fdfcc0c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd393d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc06d910 con 0x7fdfcc0d0660 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd393d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc0b1d80 con 0x7fdfcc15e6e0 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd393d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc160cd0 con 0x7fdfcc162ab0 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd16d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc0d0660 0x7fdfcc0c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd16d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc0d0660 0x7fdfcc0c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39146/0 (socket says 172.21.15.97:39146) 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd16d9640 1 -- 172.21.15.97:0/3766486488 learned_addr learned my addr 172.21.15.97:0/3766486488 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd0ed8640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc15e6e0 0x7fdfcc159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd1eda640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc162ab0 0x7fdfcc0c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd16d9640 1 -- 172.21.15.97:0/3766486488 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc162ab0 msgr2=0x7fdfcc0c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd16d9640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc162ab0 0x7fdfcc0c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd16d9640 1 -- 172.21.15.97:0/3766486488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc15e6e0 msgr2=0x7fdfcc159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd16d9640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc15e6e0 0x7fdfcc159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd16d9640 1 -- 172.21.15.97:0/3766486488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc0c8b10 con 0x7fdfcc0d0660 2024-03-29T04:07:48.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd16d9640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc0d0660 0x7fdfcc0c84a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8000f80 tx=0x7fdfc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:48.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfc1ffb640 1 -- 172.21.15.97:0/3766486488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8043020 con 0x7fdfcc0d0660 2024-03-29T04:07:48.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfc1ffb640 1 -- 172.21.15.97:0/3766486488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc802ce50 con 0x7fdfcc0d0660 2024-03-29T04:07:48.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfd393d640 1 -- 172.21.15.97:0/3766486488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc160680 con 0x7fdfcc0d0660 2024-03-29T04:07:48.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.535+0000 7fdfc1ffb640 1 -- 172.21.15.97:0/3766486488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8009510 con 0x7fdfcc0d0660 2024-03-29T04:07:48.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.536+0000 7fdfd393d640 1 -- 172.21.15.97:0/3766486488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc0c6e60 con 0x7fdfcc0d0660 2024-03-29T04:07:48.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.537+0000 7fdfd393d640 1 -- 172.21.15.97:0/3766486488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfcc068640 con 0x7fdfcc0d0660 2024-03-29T04:07:48.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.537+0000 7fdfc1ffb640 1 -- 172.21.15.97:0/3766486488 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdfc8046020 con 0x7fdfcc0d0660 2024-03-29T04:07:48.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.538+0000 7fdfc1ffb640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfb006c530 0x7fdfb006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:48.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.538+0000 7fdfd0ed8640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfb006c530 0x7fdfb006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:48.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.538+0000 7fdfc1ffb640 1 -- 172.21.15.97:0/3766486488 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdfc80adbc0 con 0x7fdfcc0d0660 2024-03-29T04:07:48.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.538+0000 7fdfd0ed8640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfb006c530 0x7fdfb006e9f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc15fa80 tx=0x7fdfbc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:48.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.542+0000 7fdfc1ffb640 1 -- 172.21.15.97:0/3766486488 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdfc804d030 con 0x7fdfcc0d0660 2024-03-29T04:07:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.716+0000 7fdfd393d640 1 -- 172.21.15.97:0/3766486488 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfcc0b4b00 con 0x7fdfb006c530 2024-03-29T04:07:48.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.722+0000 7fdfc1ffb640 1 -- 172.21.15.97:0/3766486488 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fdfcc0b4b00 con 0x7fdfb006c530 2024-03-29T04:07:48.704 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:48.706 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-29T04:07:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.725+0000 7fdfd393d640 1 -- 172.21.15.97:0/3766486488 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfb006c530 msgr2=0x7fdfb006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.725+0000 7fdfd393d640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfb006c530 0x7fdfb006e9f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc15fa80 tx=0x7fdfbc009040 comp rx=0 tx=0).stop 2024-03-29T04:07:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.725+0000 7fdfd393d640 1 -- 172.21.15.97:0/3766486488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc0d0660 msgr2=0x7fdfcc0c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.725+0000 7fdfd393d640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc0d0660 0x7fdfcc0c84a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8000f80 tx=0x7fdfc8004a40 comp rx=0 tx=0).stop 2024-03-29T04:07:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.725+0000 7fdfd393d640 1 -- 172.21.15.97:0/3766486488 shutdown_connections 2024-03-29T04:07:48.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.725+0000 7fdfd393d640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfb006c530 0x7fdfb006e9f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.725+0000 7fdfd393d640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc162ab0 0x7fdfcc0c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.725+0000 7fdfd393d640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc15e6e0 0x7fdfcc159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.725+0000 7fdfd393d640 1 --2- 172.21.15.97:0/3766486488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc0d0660 0x7fdfcc0c84a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.725+0000 7fdfd393d640 1 -- 172.21.15.97:0/3766486488 >> 172.21.15.97:0/3766486488 conn(0x7fdfcc0b3f10 msgr2=0x7fdfcc0b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:48.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.725+0000 7fdfd393d640 1 -- 172.21.15.97:0/3766486488 shutdown_connections 2024-03-29T04:07:48.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.725+0000 7fdfd393d640 1 -- 172.21.15.97:0/3766486488 wait complete. 2024-03-29T04:07:48.716 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-29T04:07:48.640081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001576"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798559+0000","last_change":"2024-03-29T04:07:41.422348+0000","last_active":"2024-03-29T04:07:42.798559+0000","last_peered":"2024-03-29T04:07:42.798559+0000","last_clean":"2024-03-29T04:07:42.798559+0000","last_became_active":"2024-03-29T04:07:41.422010+0000","last_became_peered":"2024-03-29T04:07:41.422010+0000","last_unstale":"2024-03-29T04:07:42.798559+0000","last_undegraded":"2024-03-29T04:07:42.798559+0000","last_fullsized":"2024-03-29T04:07:42.798559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:45:40.024022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413460+0000","last_change":"2024-03-29T04:07:41.426583+0000","last_active":"2024-03-29T04:07:42.413460+0000","last_peered":"2024-03-29T04:07:42.413460+0000","last_clean":"2024-03-29T04:07:42.413460+0000","last_became_active":"2024-03-29T04:07:41.426321+0000","last_became_peered":"2024-03-29T04:07:41.426321+0000","last_unstale":"2024-03-29T04:07:42.413460+0000","last_undegraded":"2024-03-29T04:07:42.413460+0000","last_fullsized":"2024-03-29T04:07:42.413460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:28:53.046219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798410+0000","last_change":"2024-03-29T04:07:41.430842+0000","last_active":"2024-03-29T04:07:42.798410+0000","last_peered":"2024-03-29T04:07:42.798410+0000","last_clean":"2024-03-29T04:07:42.798410+0000","last_became_active":"2024-03-29T04:07:41.430402+0000","last_became_peered":"2024-03-29T04:07:41.430402+0000","last_unstale":"2024-03-29T04:07:42.798410+0000","last_undegraded":"2024-03-29T04:07:42.798410+0000","last_fullsized":"2024-03-29T04:07:42.798410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:23:32.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798165+0000","last_change":"2024-03-29T04:07:41.436930+0000","last_active":"2024-03-29T04:07:42.798165+0000","last_peered":"2024-03-29T04:07:42.798165+0000","last_clean":"2024-03-29T04:07:42.798165+0000","last_became_active":"2024-03-29T04:07:41.436434+0000","last_became_peered":"2024-03-29T04:07:41.436434+0000","last_unstale":"2024-03-29T04:07:42.798165+0000","last_undegraded":"2024-03-29T04:07:42.798165+0000","last_fullsized":"2024-03-29T04:07:42.798165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:53:03.772026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798028+0000","last_change":"2024-03-29T04:07:41.428211+0000","last_active":"2024-03-29T04:07:42.798028+0000","last_peered":"2024-03-29T04:07:42.798028+0000","last_clean":"2024-03-29T04:07:42.798028+0000","last_became_active":"2024-03-29T04:07:41.427906+0000","last_became_peered":"2024-03-29T04:07:41.427906+0000","last_unstale":"2024-03-29T04:07:42.798028+0000","last_undegraded":"2024-03-29T04:07:42.798028+0000","last_fullsized":"2024-03-29T04:07:42.798028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:27:35.869429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797533+0000","last_change":"2024-03-29T04:07:41.438804+0000","last_active":"2024-03-29T04:07:42.797533+0000","last_peered":"2024-03-29T04:07:42.797533+0000","last_clean":"2024-03-29T04:07:42.797533+0000","last_became_active":"2024-03-29T04:07:41.438533+0000","last_became_peered":"2024-03-29T04:07:41.438533+0000","last_unstale":"2024-03-29T04:07:42.797533+0000","last_undegraded":"2024-03-29T04:07:42.797533+0000","last_fullsized":"2024-03-29T04:07:42.797533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:08.125575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797742+0000","last_change":"2024-03-29T04:07:41.426635+0000","last_active":"2024-03-29T04:07:42.797742+0000","last_peered":"2024-03-29T04:07:42.797742+0000","last_clean":"2024-03-29T04:07:42.797742+0000","last_became_active":"2024-03-29T04:07:41.426301+0000","last_became_peered":"2024-03-29T04:07:41.426301+0000","last_unstale":"2024-03-29T04:07:42.797742+0000","last_undegraded":"2024-03-29T04:07:42.797742+0000","last_fullsized":"2024-03-29T04:07:42.797742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:27:37.837537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798679+0000","last_change":"2024-03-29T04:07:41.436075+0000","last_active":"2024-03-29T04:07:42.798679+0000","last_peered":"2024-03-29T04:07:42.798679+0000","last_clean":"2024-03-29T04:07:42.798679+0000","last_became_active":"2024-03-29T04:07:41.435833+0000","last_became_peered":"2024-03-29T04:07:41.435833+0000","last_unstale":"2024-03-29T04:07:42.798679+0000","last_undegraded":"2024-03-29T04:07:42.798679+0000","last_fullsized":"2024-03-29T04:07:42.798679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:23:32.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798649+0000","last_change":"2024-03-29T04:07:41.441371+0000","last_active":"2024-03-29T04:07:42.798649+0000","last_peered":"2024-03-29T04:07:42.798649+0000","last_clean":"2024-03-29T04:07:42.798649+0000","last_became_active":"2024-03-29T04:07:41.441008+0000","last_became_peered":"2024-03-29T04:07:41.441008+0000","last_unstale":"2024-03-29T04:07:42.798649+0000","last_undegraded":"2024-03-29T04:07:42.798649+0000","last_fullsized":"2024-03-29T04:07:42.798649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:14:40.086642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417268+0000","last_change":"2024-03-29T04:07:41.436960+0000","last_active":"2024-03-29T04:07:42.417268+0000","last_peered":"2024-03-29T04:07:42.417268+0000","last_clean":"2024-03-29T04:07:42.417268+0000","last_became_active":"2024-03-29T04:07:41.436652+0000","last_became_peered":"2024-03-29T04:07:41.436652+0000","last_unstale":"2024-03-29T04:07:42.417268+0000","last_undegraded":"2024-03-29T04:07:42.417268+0000","last_fullsized":"2024-03-29T04:07:42.417268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:23:32.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417196+0000","last_change":"2024-03-29T04:07:41.437571+0000","last_active":"2024-03-29T04:07:42.417196+0000","last_peered":"2024-03-29T04:07:42.417196+0000","last_clean":"2024-03-29T04:07:42.417196+0000","last_became_active":"2024-03-29T04:07:41.437307+0000","last_became_peered":"2024-03-29T04:07:41.437307+0000","last_unstale":"2024-03-29T04:07:42.417196+0000","last_undegraded":"2024-03-29T04:07:42.417196+0000","last_fullsized":"2024-03-29T04:07:42.417196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:30:59.143986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417226+0000","last_change":"2024-03-29T04:07:41.429087+0000","last_active":"2024-03-29T04:07:42.417226+0000","last_peered":"2024-03-29T04:07:42.417226+0000","last_clean":"2024-03-29T04:07:42.417226+0000","last_became_active":"2024-03-29T04:07:41.427764+0000","last_became_peered":"2024-03-29T04:07:41.427764+0000","last_unstale":"2024-03-29T04:07:42.417226+0000","last_undegraded":"2024-03-29T04:07:42.417226+0000","last_fullsized":"2024-03-29T04:07:42.417226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:12.561151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.799039+0000","last_change":"2024-03-29T04:07:41.427842+0000","last_active":"2024-03-29T04:07:42.799039+0000","last_peered":"2024-03-29T04:07:42.799039+0000","last_clean":"2024-03-29T04:07:42.799039+0000","last_became_active":"2024-03-29T04:07:41.427481+0000","last_became_peered":"2024-03-29T04:07:41.427481+0000","last_unstale":"2024-03-29T04:07:42.799039+0000","last_undegraded":"2024-03-29T04:07:42.799039+0000","last_fullsized":"2024-03-29T04:07:42.799039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:53:03.772026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798076+0000","last_change":"2024-03-29T04:07:41.425389+0000","last_active":"2024-03-29T04:07:42.798076+0000","last_peered":"2024-03-29T04:07:42.798076+0000","last_clean":"2024-03-29T04:07:42.798076+0000","last_became_active":"2024-03-29T04:07:41.425016+0000","last_became_peered":"2024-03-29T04:07:41.425016+0000","last_unstale":"2024-03-29T04:07:42.798076+0000","last_undegraded":"2024-03-29T04:07:42.798076+0000","last_fullsized":"2024-03-29T04:07:42.798076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:57.438333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413581+0000","last_change":"2024-03-29T04:07:41.428282+0000","last_active":"2024-03-29T04:07:42.413581+0000","last_peered":"2024-03-29T04:07:42.413581+0000","last_clean":"2024-03-29T04:07:42.413581+0000","last_became_active":"2024-03-29T04:07:41.428042+0000","last_became_peered":"2024-03-29T04:07:41.428042+0000","last_unstale":"2024-03-29T04:07:42.413581+0000","last_undegraded":"2024-03-29T04:07:42.413581+0000","last_fullsized":"2024-03-29T04:07:42.413581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:23:32.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797464+0000","last_change":"2024-03-29T04:07:41.434962+0000","last_active":"2024-03-29T04:07:42.797464+0000","last_peered":"2024-03-29T04:07:42.797464+0000","last_clean":"2024-03-29T04:07:42.797464+0000","last_became_active":"2024-03-29T04:07:41.434691+0000","last_became_peered":"2024-03-29T04:07:41.434691+0000","last_unstale":"2024-03-29T04:07:42.797464+0000","last_undegraded":"2024-03-29T04:07:42.797464+0000","last_fullsized":"2024-03-29T04:07:42.797464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:23:32.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798581+0000","last_change":"2024-03-29T04:07:41.434716+0000","last_active":"2024-03-29T04:07:42.798581+0000","last_peered":"2024-03-29T04:07:42.798581+0000","last_clean":"2024-03-29T04:07:42.798581+0000","last_became_active":"2024-03-29T04:07:41.434372+0000","last_became_peered":"2024-03-29T04:07:41.434372+0000","last_unstale":"2024-03-29T04:07:42.798581+0000","last_undegraded":"2024-03-29T04:07:42.798581+0000","last_fullsized":"2024-03-29T04:07:42.798581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:14.560489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797999+0000","last_change":"2024-03-29T04:07:41.427613+0000","last_active":"2024-03-29T04:07:42.797999+0000","last_peered":"2024-03-29T04:07:42.797999+0000","last_clean":"2024-03-29T04:07:42.797999+0000","last_became_active":"2024-03-29T04:07:41.427338+0000","last_became_peered":"2024-03-29T04:07:41.427338+0000","last_unstale":"2024-03-29T04:07:42.797999+0000","last_undegraded":"2024-03-29T04:07:42.797999+0000","last_fullsized":"2024-03-29T04:07:42.797999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:42.838588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798413+0000","last_change":"2024-03-29T04:07:41.427942+0000","last_active":"2024-03-29T04:07:42.798413+0000","last_peered":"2024-03-29T04:07:42.798413+0000","last_clean":"2024-03-29T04:07:42.798413+0000","last_became_active":"2024-03-29T04:07:41.427667+0000","last_became_peered":"2024-03-29T04:07:41.427667+0000","last_unstale":"2024-03-29T04:07:42.798413+0000","last_undegraded":"2024-03-29T04:07:42.798413+0000","last_fullsized":"2024-03-29T04:07:42.798413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:12:50.176893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417255+0000","last_change":"2024-03-29T04:07:41.429646+0000","last_active":"2024-03-29T04:07:42.417255+0000","last_peered":"2024-03-29T04:07:42.417255+0000","last_clean":"2024-03-29T04:07:42.417255+0000","last_became_active":"2024-03-29T04:07:41.429376+0000","last_became_peered":"2024-03-29T04:07:41.429376+0000","last_unstale":"2024-03-29T04:07:42.417255+0000","last_undegraded":"2024-03-29T04:07:42.417255+0000","last_fullsized":"2024-03-29T04:07:42.417255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:35.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797955+0000","last_change":"2024-03-29T04:07:41.425314+0000","last_active":"2024-03-29T04:07:42.797955+0000","last_peered":"2024-03-29T04:07:42.797955+0000","last_clean":"2024-03-29T04:07:42.797955+0000","last_became_active":"2024-03-29T04:07:41.424897+0000","last_became_peered":"2024-03-29T04:07:41.424897+0000","last_unstale":"2024-03-29T04:07:42.797955+0000","last_undegraded":"2024-03-29T04:07:42.797955+0000","last_fullsized":"2024-03-29T04:07:42.797955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:31:27.159107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797567+0000","last_change":"2024-03-29T04:07:41.441556+0000","last_active":"2024-03-29T04:07:42.797567+0000","last_peered":"2024-03-29T04:07:42.797567+0000","last_clean":"2024-03-29T04:07:42.797567+0000","last_became_active":"2024-03-29T04:07:41.437251+0000","last_became_peered":"2024-03-29T04:07:41.437251+0000","last_unstale":"2024-03-29T04:07:42.797567+0000","last_undegraded":"2024-03-29T04:07:42.797567+0000","last_fullsized":"2024-03-29T04:07:42.797567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:32:16.919913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798200+0000","last_change":"2024-03-29T04:07:41.435226+0000","last_active":"2024-03-29T04:07:42.798200+0000","last_peered":"2024-03-29T04:07:42.798200+0000","last_clean":"2024-03-29T04:07:42.798200+0000","last_became_active":"2024-03-29T04:07:41.434543+0000","last_became_peered":"2024-03-29T04:07:41.434543+0000","last_unstale":"2024-03-29T04:07:42.798200+0000","last_undegraded":"2024-03-29T04:07:42.798200+0000","last_fullsized":"2024-03-29T04:07:42.798200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:30:59.143986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797501+0000","last_change":"2024-03-29T04:07:41.434893+0000","last_active":"2024-03-29T04:07:42.797501+0000","last_peered":"2024-03-29T04:07:42.797501+0000","last_clean":"2024-03-29T04:07:42.797501+0000","last_became_active":"2024-03-29T04:07:41.434578+0000","last_became_peered":"2024-03-29T04:07:41.434578+0000","last_unstale":"2024-03-29T04:07:42.797501+0000","last_undegraded":"2024-03-29T04:07:42.797501+0000","last_fullsized":"2024-03-29T04:07:42.797501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:40.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798261+0000","last_change":"2024-03-29T04:07:41.434765+0000","last_active":"2024-03-29T04:07:42.798261+0000","last_peered":"2024-03-29T04:07:42.798261+0000","last_clean":"2024-03-29T04:07:42.798261+0000","last_became_active":"2024-03-29T04:07:41.434463+0000","last_became_peered":"2024-03-29T04:07:41.434463+0000","last_unstale":"2024-03-29T04:07:42.798261+0000","last_undegraded":"2024-03-29T04:07:42.798261+0000","last_fullsized":"2024-03-29T04:07:42.798261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:28:53.046219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417218+0000","last_change":"2024-03-29T04:07:41.424421+0000","last_active":"2024-03-29T04:07:42.417218+0000","last_peered":"2024-03-29T04:07:42.417218+0000","last_clean":"2024-03-29T04:07:42.417218+0000","last_became_active":"2024-03-29T04:07:41.424154+0000","last_became_peered":"2024-03-29T04:07:41.424154+0000","last_unstale":"2024-03-29T04:07:42.417218+0000","last_undegraded":"2024-03-29T04:07:42.417218+0000","last_fullsized":"2024-03-29T04:07:42.417218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:28:53.046219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413456+0000","last_change":"2024-03-29T04:07:41.429051+0000","last_active":"2024-03-29T04:07:42.413456+0000","last_peered":"2024-03-29T04:07:42.413456+0000","last_clean":"2024-03-29T04:07:42.413456+0000","last_became_active":"2024-03-29T04:07:41.428748+0000","last_became_peered":"2024-03-29T04:07:41.428748+0000","last_unstale":"2024-03-29T04:07:42.413456+0000","last_undegraded":"2024-03-29T04:07:42.413456+0000","last_fullsized":"2024-03-29T04:07:42.413456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:30:59.143986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798736+0000","last_change":"2024-03-29T04:07:41.435463+0000","last_active":"2024-03-29T04:07:42.798736+0000","last_peered":"2024-03-29T04:07:42.798736+0000","last_clean":"2024-03-29T04:07:42.798736+0000","last_became_active":"2024-03-29T04:07:41.434902+0000","last_became_peered":"2024-03-29T04:07:41.434902+0000","last_unstale":"2024-03-29T04:07:42.798736+0000","last_undegraded":"2024-03-29T04:07:42.798736+0000","last_fullsized":"2024-03-29T04:07:42.798736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:40.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798097+0000","last_change":"2024-03-29T04:07:41.428583+0000","last_active":"2024-03-29T04:07:42.798097+0000","last_peered":"2024-03-29T04:07:42.798097+0000","last_clean":"2024-03-29T04:07:42.798097+0000","last_became_active":"2024-03-29T04:07:41.428264+0000","last_became_peered":"2024-03-29T04:07:41.428264+0000","last_unstale":"2024-03-29T04:07:42.798097+0000","last_undegraded":"2024-03-29T04:07:42.798097+0000","last_fullsized":"2024-03-29T04:07:42.798097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:30:59.143986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797685+0000","last_change":"2024-03-29T04:07:41.433555+0000","last_active":"2024-03-29T04:07:42.797685+0000","last_peered":"2024-03-29T04:07:42.797685+0000","last_clean":"2024-03-29T04:07:42.797685+0000","last_became_active":"2024-03-29T04:07:41.433253+0000","last_became_peered":"2024-03-29T04:07:41.433253+0000","last_unstale":"2024-03-29T04:07:42.797685+0000","last_undegraded":"2024-03-29T04:07:42.797685+0000","last_fullsized":"2024-03-29T04:07:42.797685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:27:37.837537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797593+0000","last_change":"2024-03-29T04:07:41.433903+0000","last_active":"2024-03-29T04:07:42.797593+0000","last_peered":"2024-03-29T04:07:42.797593+0000","last_clean":"2024-03-29T04:07:42.797593+0000","last_became_active":"2024-03-29T04:07:41.433582+0000","last_became_peered":"2024-03-29T04:07:41.433582+0000","last_unstale":"2024-03-29T04:07:42.797593+0000","last_undegraded":"2024-03-29T04:07:42.797593+0000","last_fullsized":"2024-03-29T04:07:42.797593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:45.683611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417165+0000","last_change":"2024-03-29T04:07:41.426138+0000","last_active":"2024-03-29T04:07:42.417165+0000","last_peered":"2024-03-29T04:07:42.417165+0000","last_clean":"2024-03-29T04:07:42.417165+0000","last_became_active":"2024-03-29T04:07:41.425636+0000","last_became_peered":"2024-03-29T04:07:41.425636+0000","last_unstale":"2024-03-29T04:07:42.417165+0000","last_undegraded":"2024-03-29T04:07:42.417165+0000","last_fullsized":"2024-03-29T04:07:42.417165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:31:27.159107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417281+0000","last_change":"2024-03-29T04:07:41.434449+0000","last_active":"2024-03-29T04:07:42.417281+0000","last_peered":"2024-03-29T04:07:42.417281+0000","last_clean":"2024-03-29T04:07:42.417281+0000","last_became_active":"2024-03-29T04:07:41.434169+0000","last_became_peered":"2024-03-29T04:07:41.434169+0000","last_unstale":"2024-03-29T04:07:42.417281+0000","last_undegraded":"2024-03-29T04:07:42.417281+0000","last_fullsized":"2024-03-29T04:07:42.417281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:35.964938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.799089+0000","last_change":"2024-03-29T04:07:40.425200+0000","last_active":"2024-03-29T04:07:42.799089+0000","last_peered":"2024-03-29T04:07:42.799089+0000","last_clean":"2024-03-29T04:07:42.799089+0000","last_became_active":"2024-03-29T04:07:40.424061+0000","last_became_peered":"2024-03-29T04:07:40.424061+0000","last_unstale":"2024-03-29T04:07:42.799089+0000","last_undegraded":"2024-03-29T04:07:42.799089+0000","last_fullsized":"2024-03-29T04:07:42.799089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:04.924835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798235+0000","last_change":"2024-03-29T04:07:41.432958+0000","last_active":"2024-03-29T04:07:42.798235+0000","last_peered":"2024-03-29T04:07:42.798235+0000","last_clean":"2024-03-29T04:07:42.798235+0000","last_became_active":"2024-03-29T04:07:41.432654+0000","last_became_peered":"2024-03-29T04:07:41.432654+0000","last_unstale":"2024-03-29T04:07:42.798235+0000","last_undegraded":"2024-03-29T04:07:42.798235+0000","last_fullsized":"2024-03-29T04:07:42.798235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:35.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.416646+0000","last_change":"2024-03-29T04:07:40.418823+0000","last_active":"2024-03-29T04:07:42.416646+0000","last_peered":"2024-03-29T04:07:42.416646+0000","last_clean":"2024-03-29T04:07:42.416646+0000","last_became_active":"2024-03-29T04:07:40.418542+0000","last_became_peered":"2024-03-29T04:07:40.418542+0000","last_unstale":"2024-03-29T04:07:42.416646+0000","last_undegraded":"2024-03-29T04:07:42.416646+0000","last_fullsized":"2024-03-29T04:07:42.416646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:04.924835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798353+0000","last_change":"2024-03-29T04:07:40.430532+0000","last_active":"2024-03-29T04:07:42.798353+0000","last_peered":"2024-03-29T04:07:42.798353+0000","last_clean":"2024-03-29T04:07:42.798353+0000","last_became_active":"2024-03-29T04:07:40.430321+0000","last_became_peered":"2024-03-29T04:07:40.430321+0000","last_unstale":"2024-03-29T04:07:42.798353+0000","last_undegraded":"2024-03-29T04:07:42.798353+0000","last_fullsized":"2024-03-29T04:07:42.798353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:40:12.105363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798804+0000","last_change":"2024-03-29T04:07:41.426187+0000","last_active":"2024-03-29T04:07:42.798804+0000","last_peered":"2024-03-29T04:07:42.798804+0000","last_clean":"2024-03-29T04:07:42.798804+0000","last_became_active":"2024-03-29T04:07:41.425926+0000","last_became_peered":"2024-03-29T04:07:41.425926+0000","last_unstale":"2024-03-29T04:07:42.798804+0000","last_undegraded":"2024-03-29T04:07:42.798804+0000","last_fullsized":"2024-03-29T04:07:42.798804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:08.125575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797558+0000","last_change":"2024-03-29T04:07:40.430187+0000","last_active":"2024-03-29T04:07:42.797558+0000","last_peered":"2024-03-29T04:07:42.797558+0000","last_clean":"2024-03-29T04:07:42.797558+0000","last_became_active":"2024-03-29T04:07:40.429878+0000","last_became_peered":"2024-03-29T04:07:40.429878+0000","last_unstale":"2024-03-29T04:07:42.797558+0000","last_undegraded":"2024-03-29T04:07:42.797558+0000","last_fullsized":"2024-03-29T04:07:42.797558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413340+0000","last_change":"2024-03-29T04:07:41.427276+0000","last_active":"2024-03-29T04:07:42.413340+0000","last_peered":"2024-03-29T04:07:42.413340+0000","last_clean":"2024-03-29T04:07:42.413340+0000","last_became_active":"2024-03-29T04:07:41.427019+0000","last_became_peered":"2024-03-29T04:07:41.427019+0000","last_unstale":"2024-03-29T04:07:42.413340+0000","last_undegraded":"2024-03-29T04:07:42.413340+0000","last_fullsized":"2024-03-29T04:07:42.413340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:31:27.159107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.416916+0000","last_change":"2024-03-29T04:07:40.430476+0000","last_active":"2024-03-29T04:07:42.416916+0000","last_peered":"2024-03-29T04:07:42.416916+0000","last_clean":"2024-03-29T04:07:42.416916+0000","last_became_active":"2024-03-29T04:07:40.430169+0000","last_became_peered":"2024-03-29T04:07:40.430169+0000","last_unstale":"2024-03-29T04:07:42.416916+0000","last_undegraded":"2024-03-29T04:07:42.416916+0000","last_fullsized":"2024-03-29T04:07:42.416916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798544+0000","last_change":"2024-03-29T04:07:41.439222+0000","last_active":"2024-03-29T04:07:42.798544+0000","last_peered":"2024-03-29T04:07:42.798544+0000","last_clean":"2024-03-29T04:07:42.798544+0000","last_became_active":"2024-03-29T04:07:41.437292+0000","last_became_peered":"2024-03-29T04:07:41.437292+0000","last_unstale":"2024-03-29T04:07:42.798544+0000","last_undegraded":"2024-03-29T04:07:42.798544+0000","last_fullsized":"2024-03-29T04:07:42.798544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:13:07.317381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798565+0000","last_change":"2024-03-29T04:07:40.425014+0000","last_active":"2024-03-29T04:07:42.798565+0000","last_peered":"2024-03-29T04:07:42.798565+0000","last_clean":"2024-03-29T04:07:42.798565+0000","last_became_active":"2024-03-29T04:07:40.424254+0000","last_became_peered":"2024-03-29T04:07:40.424254+0000","last_unstale":"2024-03-29T04:07:42.798565+0000","last_undegraded":"2024-03-29T04:07:42.798565+0000","last_fullsized":"2024-03-29T04:07:42.798565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:08.909004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797721+0000","last_change":"2024-03-29T04:07:41.436336+0000","last_active":"2024-03-29T04:07:42.797721+0000","last_peered":"2024-03-29T04:07:42.797721+0000","last_clean":"2024-03-29T04:07:42.797721+0000","last_became_active":"2024-03-29T04:07:41.436089+0000","last_became_peered":"2024-03-29T04:07:41.436089+0000","last_unstale":"2024-03-29T04:07:42.797721+0000","last_undegraded":"2024-03-29T04:07:42.797721+0000","last_fullsized":"2024-03-29T04:07:42.797721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:19.861306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413545+0000","last_change":"2024-03-29T04:07:40.424800+0000","last_active":"2024-03-29T04:07:42.413545+0000","last_peered":"2024-03-29T04:07:42.413545+0000","last_clean":"2024-03-29T04:07:42.413545+0000","last_became_active":"2024-03-29T04:07:40.424508+0000","last_became_peered":"2024-03-29T04:07:40.424508+0000","last_unstale":"2024-03-29T04:07:42.413545+0000","last_undegraded":"2024-03-29T04:07:42.413545+0000","last_fullsized":"2024-03-29T04:07:42.413545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:44.678749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417383+0000","last_change":"2024-03-29T04:07:41.429324+0000","last_active":"2024-03-29T04:07:42.417383+0000","last_peered":"2024-03-29T04:07:42.417383+0000","last_clean":"2024-03-29T04:07:42.417383+0000","last_became_active":"2024-03-29T04:07:41.428117+0000","last_became_peered":"2024-03-29T04:07:41.428117+0000","last_unstale":"2024-03-29T04:07:42.417383+0000","last_undegraded":"2024-03-29T04:07:42.417383+0000","last_fullsized":"2024-03-29T04:07:42.417383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:45.683611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417360+0000","last_change":"2024-03-29T04:07:40.428764+0000","last_active":"2024-03-29T04:07:42.417360+0000","last_peered":"2024-03-29T04:07:42.417360+0000","last_clean":"2024-03-29T04:07:42.417360+0000","last_became_active":"2024-03-29T04:07:40.428510+0000","last_became_peered":"2024-03-29T04:07:40.428510+0000","last_unstale":"2024-03-29T04:07:42.417360+0000","last_undegraded":"2024-03-29T04:07:42.417360+0000","last_fullsized":"2024-03-29T04:07:42.417360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:34.960075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798009+0000","last_change":"2024-03-29T04:07:41.431774+0000","last_active":"2024-03-29T04:07:42.798009+0000","last_peered":"2024-03-29T04:07:42.798009+0000","last_clean":"2024-03-29T04:07:42.798009+0000","last_became_active":"2024-03-29T04:07:41.431503+0000","last_became_peered":"2024-03-29T04:07:41.431503+0000","last_unstale":"2024-03-29T04:07:42.798009+0000","last_undegraded":"2024-03-29T04:07:42.798009+0000","last_fullsized":"2024-03-29T04:07:42.798009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:53:03.772026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798322+0000","last_change":"2024-03-29T04:07:40.418935+0000","last_active":"2024-03-29T04:07:42.798322+0000","last_peered":"2024-03-29T04:07:42.798322+0000","last_clean":"2024-03-29T04:07:42.798322+0000","last_became_active":"2024-03-29T04:07:40.418678+0000","last_became_peered":"2024-03-29T04:07:40.418678+0000","last_unstale":"2024-03-29T04:07:42.798322+0000","last_undegraded":"2024-03-29T04:07:42.798322+0000","last_fullsized":"2024-03-29T04:07:42.798322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:06:18.089762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798208+0000","last_change":"2024-03-29T04:07:41.429488+0000","last_active":"2024-03-29T04:07:42.798208+0000","last_peered":"2024-03-29T04:07:42.798208+0000","last_clean":"2024-03-29T04:07:42.798208+0000","last_became_active":"2024-03-29T04:07:41.429233+0000","last_became_peered":"2024-03-29T04:07:41.429233+0000","last_unstale":"2024-03-29T04:07:42.798208+0000","last_undegraded":"2024-03-29T04:07:42.798208+0000","last_fullsized":"2024-03-29T04:07:42.798208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:28:53.046219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797793+0000","last_change":"2024-03-29T04:07:40.429229+0000","last_active":"2024-03-29T04:07:42.797793+0000","last_peered":"2024-03-29T04:07:42.797793+0000","last_clean":"2024-03-29T04:07:42.797793+0000","last_became_active":"2024-03-29T04:07:40.428949+0000","last_became_peered":"2024-03-29T04:07:40.428949+0000","last_unstale":"2024-03-29T04:07:42.797793+0000","last_undegraded":"2024-03-29T04:07:42.797793+0000","last_fullsized":"2024-03-29T04:07:42.797793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:11.556289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798285+0000","last_change":"2024-03-29T04:07:41.433525+0000","last_active":"2024-03-29T04:07:42.798285+0000","last_peered":"2024-03-29T04:07:42.798285+0000","last_clean":"2024-03-29T04:07:42.798285+0000","last_became_active":"2024-03-29T04:07:41.433128+0000","last_became_peered":"2024-03-29T04:07:41.433128+0000","last_unstale":"2024-03-29T04:07:42.798285+0000","last_undegraded":"2024-03-29T04:07:42.798285+0000","last_fullsized":"2024-03-29T04:07:42.798285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:14.560489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.416530+0000","last_change":"2024-03-29T04:07:40.419079+0000","last_active":"2024-03-29T04:07:42.416530+0000","last_peered":"2024-03-29T04:07:42.416530+0000","last_clean":"2024-03-29T04:07:42.416530+0000","last_became_active":"2024-03-29T04:07:40.418589+0000","last_became_peered":"2024-03-29T04:07:40.418589+0000","last_unstale":"2024-03-29T04:07:42.416530+0000","last_undegraded":"2024-03-29T04:07:42.416530+0000","last_fullsized":"2024-03-29T04:07:42.416530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:11.556289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417165+0000","last_change":"2024-03-29T04:07:41.425671+0000","last_active":"2024-03-29T04:07:42.417165+0000","last_peered":"2024-03-29T04:07:42.417165+0000","last_clean":"2024-03-29T04:07:42.417165+0000","last_became_active":"2024-03-29T04:07:41.425336+0000","last_became_peered":"2024-03-29T04:07:41.425336+0000","last_unstale":"2024-03-29T04:07:42.417165+0000","last_undegraded":"2024-03-29T04:07:42.417165+0000","last_fullsized":"2024-03-29T04:07:42.417165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:10:59.508554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797485+0000","last_change":"2024-03-29T04:07:40.419997+0000","last_active":"2024-03-29T04:07:42.797485+0000","last_peered":"2024-03-29T04:07:42.797485+0000","last_clean":"2024-03-29T04:07:42.797485+0000","last_became_active":"2024-03-29T04:07:40.419725+0000","last_became_peered":"2024-03-29T04:07:40.419725+0000","last_unstale":"2024-03-29T04:07:42.797485+0000","last_undegraded":"2024-03-29T04:07:42.797485+0000","last_fullsized":"2024-03-29T04:07:42.797485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:40:12.105363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797775+0000","last_change":"2024-03-29T04:07:41.429335+0000","last_active":"2024-03-29T04:07:42.797775+0000","last_peered":"2024-03-29T04:07:42.797775+0000","last_clean":"2024-03-29T04:07:42.797775+0000","last_became_active":"2024-03-29T04:07:41.428990+0000","last_became_peered":"2024-03-29T04:07:41.428990+0000","last_unstale":"2024-03-29T04:07:42.797775+0000","last_undegraded":"2024-03-29T04:07:42.797775+0000","last_fullsized":"2024-03-29T04:07:42.797775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:58.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.418057+0000","last_change":"2024-03-29T04:07:40.422355+0000","last_active":"2024-03-29T04:07:42.418057+0000","last_peered":"2024-03-29T04:07:42.418057+0000","last_clean":"2024-03-29T04:07:42.418057+0000","last_became_active":"2024-03-29T04:07:40.422081+0000","last_became_peered":"2024-03-29T04:07:40.422081+0000","last_unstale":"2024-03-29T04:07:42.418057+0000","last_undegraded":"2024-03-29T04:07:42.418057+0000","last_fullsized":"2024-03-29T04:07:42.418057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:42.944057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798226+0000","last_change":"2024-03-29T04:07:41.428540+0000","last_active":"2024-03-29T04:07:42.798226+0000","last_peered":"2024-03-29T04:07:42.798226+0000","last_clean":"2024-03-29T04:07:42.798226+0000","last_became_active":"2024-03-29T04:07:41.428266+0000","last_became_peered":"2024-03-29T04:07:41.428266+0000","last_unstale":"2024-03-29T04:07:42.798226+0000","last_undegraded":"2024-03-29T04:07:42.798226+0000","last_fullsized":"2024-03-29T04:07:42.798226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:13:07.317381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797957+0000","last_change":"2024-03-29T04:07:40.423940+0000","last_active":"2024-03-29T04:07:42.797957+0000","last_peered":"2024-03-29T04:07:42.797957+0000","last_clean":"2024-03-29T04:07:42.797957+0000","last_became_active":"2024-03-29T04:07:40.423675+0000","last_became_peered":"2024-03-29T04:07:40.423675+0000","last_unstale":"2024-03-29T04:07:42.797957+0000","last_undegraded":"2024-03-29T04:07:42.797957+0000","last_fullsized":"2024-03-29T04:07:42.797957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:08.909004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798698+0000","last_change":"2024-03-29T04:07:41.429372+0000","last_active":"2024-03-29T04:07:42.798698+0000","last_peered":"2024-03-29T04:07:42.798698+0000","last_clean":"2024-03-29T04:07:42.798698+0000","last_became_active":"2024-03-29T04:07:41.429084+0000","last_became_peered":"2024-03-29T04:07:41.429084+0000","last_unstale":"2024-03-29T04:07:42.798698+0000","last_undegraded":"2024-03-29T04:07:42.798698+0000","last_fullsized":"2024-03-29T04:07:42.798698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:44:51.240215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413552+0000","last_change":"2024-03-29T04:07:40.420126+0000","last_active":"2024-03-29T04:07:42.413552+0000","last_peered":"2024-03-29T04:07:42.413552+0000","last_clean":"2024-03-29T04:07:42.413552+0000","last_became_active":"2024-03-29T04:07:40.419852+0000","last_became_peered":"2024-03-29T04:07:40.419852+0000","last_unstale":"2024-03-29T04:07:42.413552+0000","last_undegraded":"2024-03-29T04:07:42.413552+0000","last_fullsized":"2024-03-29T04:07:42.413552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:41.833725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798305+0000","last_change":"2024-03-29T04:07:41.435921+0000","last_active":"2024-03-29T04:07:42.798305+0000","last_peered":"2024-03-29T04:07:42.798305+0000","last_clean":"2024-03-29T04:07:42.798305+0000","last_became_active":"2024-03-29T04:07:41.435681+0000","last_became_peered":"2024-03-29T04:07:41.435681+0000","last_unstale":"2024-03-29T04:07:42.798305+0000","last_undegraded":"2024-03-29T04:07:42.798305+0000","last_fullsized":"2024-03-29T04:07:42.798305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:28:45.871462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797831+0000","last_change":"2024-03-29T04:07:40.430102+0000","last_active":"2024-03-29T04:07:42.797831+0000","last_peered":"2024-03-29T04:07:42.797831+0000","last_clean":"2024-03-29T04:07:42.797831+0000","last_became_active":"2024-03-29T04:07:40.429846+0000","last_became_peered":"2024-03-29T04:07:40.429846+0000","last_unstale":"2024-03-29T04:07:42.797831+0000","last_undegraded":"2024-03-29T04:07:42.797831+0000","last_fullsized":"2024-03-29T04:07:42.797831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798212+0000","last_change":"2024-03-29T04:07:41.438935+0000","last_active":"2024-03-29T04:07:42.798212+0000","last_peered":"2024-03-29T04:07:42.798212+0000","last_clean":"2024-03-29T04:07:42.798212+0000","last_became_active":"2024-03-29T04:07:41.438629+0000","last_became_peered":"2024-03-29T04:07:41.438629+0000","last_unstale":"2024-03-29T04:07:42.798212+0000","last_undegraded":"2024-03-29T04:07:42.798212+0000","last_fullsized":"2024-03-29T04:07:42.798212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:28:45.871462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797615+0000","last_change":"2024-03-29T04:07:29.856799+0000","last_active":"2024-03-29T04:07:42.797615+0000","last_peered":"2024-03-29T04:07:42.797615+0000","last_clean":"2024-03-29T04:07:42.797615+0000","last_became_active":"2024-03-29T04:07:29.855873+0000","last_became_peered":"2024-03-29T04:07:29.855873+0000","last_unstale":"2024-03-29T04:07:42.797615+0000","last_undegraded":"2024-03-29T04:07:42.797615+0000","last_fullsized":"2024-03-29T04:07:42.797615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_clean_scrub_stamp":"2024-03-29T04:07:28.661516+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:25:15.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797863+0000","last_change":"2024-03-29T04:07:40.427444+0000","last_active":"2024-03-29T04:07:42.797863+0000","last_peered":"2024-03-29T04:07:42.797863+0000","last_clean":"2024-03-29T04:07:42.797863+0000","last_became_active":"2024-03-29T04:07:40.426408+0000","last_became_peered":"2024-03-29T04:07:40.426408+0000","last_unstale":"2024-03-29T04:07:42.797863+0000","last_undegraded":"2024-03-29T04:07:42.797863+0000","last_fullsized":"2024-03-29T04:07:42.797863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:04.924835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798093+0000","last_change":"2024-03-29T04:07:41.441256+0000","last_active":"2024-03-29T04:07:42.798093+0000","last_peered":"2024-03-29T04:07:42.798093+0000","last_clean":"2024-03-29T04:07:42.798093+0000","last_became_active":"2024-03-29T04:07:41.441002+0000","last_became_peered":"2024-03-29T04:07:41.441002+0000","last_unstale":"2024-03-29T04:07:42.798093+0000","last_undegraded":"2024-03-29T04:07:42.798093+0000","last_fullsized":"2024-03-29T04:07:42.798093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:57.438333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797519+0000","last_change":"2024-03-29T04:07:40.427239+0000","last_active":"2024-03-29T04:07:42.797519+0000","last_peered":"2024-03-29T04:07:42.797519+0000","last_clean":"2024-03-29T04:07:42.797519+0000","last_became_active":"2024-03-29T04:07:40.426943+0000","last_became_peered":"2024-03-29T04:07:40.426943+0000","last_unstale":"2024-03-29T04:07:42.797519+0000","last_undegraded":"2024-03-29T04:07:42.797519+0000","last_fullsized":"2024-03-29T04:07:42.797519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:50:50.615524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797844+0000","last_change":"2024-03-29T04:07:41.433014+0000","last_active":"2024-03-29T04:07:42.797844+0000","last_peered":"2024-03-29T04:07:42.797844+0000","last_clean":"2024-03-29T04:07:42.797844+0000","last_became_active":"2024-03-29T04:07:41.432678+0000","last_became_peered":"2024-03-29T04:07:41.432678+0000","last_unstale":"2024-03-29T04:07:42.797844+0000","last_undegraded":"2024-03-29T04:07:42.797844+0000","last_fullsized":"2024-03-29T04:07:42.797844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:10:59.508554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797581+0000","last_change":"2024-03-29T04:07:40.421624+0000","last_active":"2024-03-29T04:07:42.797581+0000","last_peered":"2024-03-29T04:07:42.797581+0000","last_clean":"2024-03-29T04:07:42.797581+0000","last_became_active":"2024-03-29T04:07:40.421376+0000","last_became_peered":"2024-03-29T04:07:40.421376+0000","last_unstale":"2024-03-29T04:07:42.797581+0000","last_undegraded":"2024-03-29T04:07:42.797581+0000","last_fullsized":"2024-03-29T04:07:42.797581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:04.924835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413533+0000","last_change":"2024-03-29T04:07:41.421588+0000","last_active":"2024-03-29T04:07:42.413533+0000","last_peered":"2024-03-29T04:07:42.413533+0000","last_clean":"2024-03-29T04:07:42.413533+0000","last_became_active":"2024-03-29T04:07:41.421329+0000","last_became_peered":"2024-03-29T04:07:41.421329+0000","last_unstale":"2024-03-29T04:07:42.413533+0000","last_undegraded":"2024-03-29T04:07:42.413533+0000","last_fullsized":"2024-03-29T04:07:42.413533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:35.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797840+0000","last_change":"2024-03-29T04:07:40.424596+0000","last_active":"2024-03-29T04:07:42.797840+0000","last_peered":"2024-03-29T04:07:42.797840+0000","last_clean":"2024-03-29T04:07:42.797840+0000","last_became_active":"2024-03-29T04:07:40.424288+0000","last_became_peered":"2024-03-29T04:07:40.424288+0000","last_unstale":"2024-03-29T04:07:42.797840+0000","last_undegraded":"2024-03-29T04:07:42.797840+0000","last_fullsized":"2024-03-29T04:07:42.797840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:34.960075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.418058+0000","last_change":"2024-03-29T04:07:41.429654+0000","last_active":"2024-03-29T04:07:42.418058+0000","last_peered":"2024-03-29T04:07:42.418058+0000","last_clean":"2024-03-29T04:07:42.418058+0000","last_became_active":"2024-03-29T04:07:41.429335+0000","last_became_peered":"2024-03-29T04:07:41.429335+0000","last_unstale":"2024-03-29T04:07:42.418058+0000","last_undegraded":"2024-03-29T04:07:42.418058+0000","last_fullsized":"2024-03-29T04:07:42.418058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:50:51.620387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413527+0000","last_change":"2024-03-29T04:07:40.420207+0000","last_active":"2024-03-29T04:07:42.413527+0000","last_peered":"2024-03-29T04:07:42.413527+0000","last_clean":"2024-03-29T04:07:42.413527+0000","last_became_active":"2024-03-29T04:07:40.419936+0000","last_became_peered":"2024-03-29T04:07:40.419936+0000","last_unstale":"2024-03-29T04:07:42.413527+0000","last_undegraded":"2024-03-29T04:07:42.413527+0000","last_fullsized":"2024-03-29T04:07:42.413527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:11.556289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798302+0000","last_change":"2024-03-29T04:07:41.433871+0000","last_active":"2024-03-29T04:07:42.798302+0000","last_peered":"2024-03-29T04:07:42.798302+0000","last_clean":"2024-03-29T04:07:42.798302+0000","last_became_active":"2024-03-29T04:07:41.433183+0000","last_became_peered":"2024-03-29T04:07:41.433183+0000","last_unstale":"2024-03-29T04:07:42.798302+0000","last_undegraded":"2024-03-29T04:07:42.798302+0000","last_fullsized":"2024-03-29T04:07:42.798302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:27:35.869429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798386+0000","last_change":"2024-03-29T04:07:40.432156+0000","last_active":"2024-03-29T04:07:42.798386+0000","last_peered":"2024-03-29T04:07:42.798386+0000","last_clean":"2024-03-29T04:07:42.798386+0000","last_became_active":"2024-03-29T04:07:40.431840+0000","last_became_peered":"2024-03-29T04:07:40.431840+0000","last_unstale":"2024-03-29T04:07:42.798386+0000","last_undegraded":"2024-03-29T04:07:42.798386+0000","last_fullsized":"2024-03-29T04:07:42.798386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:36:02.627626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413428+0000","last_change":"2024-03-29T04:07:41.421216+0000","last_active":"2024-03-29T04:07:42.413428+0000","last_peered":"2024-03-29T04:07:42.413428+0000","last_clean":"2024-03-29T04:07:42.413428+0000","last_became_active":"2024-03-29T04:07:41.420934+0000","last_became_peered":"2024-03-29T04:07:41.420934+0000","last_unstale":"2024-03-29T04:07:42.413428+0000","last_undegraded":"2024-03-29T04:07:42.413428+0000","last_fullsized":"2024-03-29T04:07:42.413428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:57.438333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.433094+0000","last_change":"2024-03-29T04:07:40.421842+0000","last_active":"2024-03-29T04:07:42.433094+0000","last_peered":"2024-03-29T04:07:42.433094+0000","last_clean":"2024-03-29T04:07:42.433094+0000","last_became_active":"2024-03-29T04:07:40.421590+0000","last_became_peered":"2024-03-29T04:07:40.421590+0000","last_unstale":"2024-03-29T04:07:42.433094+0000","last_undegraded":"2024-03-29T04:07:42.433094+0000","last_fullsized":"2024-03-29T04:07:42.433094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:08.909004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798399+0000","last_change":"2024-03-29T04:07:41.429458+0000","last_active":"2024-03-29T04:07:42.798399+0000","last_peered":"2024-03-29T04:07:42.798399+0000","last_clean":"2024-03-29T04:07:42.798399+0000","last_became_active":"2024-03-29T04:07:41.429115+0000","last_became_peered":"2024-03-29T04:07:41.429115+0000","last_unstale":"2024-03-29T04:07:42.798399+0000","last_undegraded":"2024-03-29T04:07:42.798399+0000","last_fullsized":"2024-03-29T04:07:42.798399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:35.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417036+0000","last_change":"2024-03-29T04:07:40.422954+0000","last_active":"2024-03-29T04:07:42.417036+0000","last_peered":"2024-03-29T04:07:42.417036+0000","last_clean":"2024-03-29T04:07:42.417036+0000","last_became_active":"2024-03-29T04:07:40.422548+0000","last_became_peered":"2024-03-29T04:07:40.422548+0000","last_unstale":"2024-03-29T04:07:42.417036+0000","last_undegraded":"2024-03-29T04:07:42.417036+0000","last_fullsized":"2024-03-29T04:07:42.417036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:08.909004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798579+0000","last_change":"2024-03-29T04:07:41.425089+0000","last_active":"2024-03-29T04:07:42.798579+0000","last_peered":"2024-03-29T04:07:42.798579+0000","last_clean":"2024-03-29T04:07:42.798579+0000","last_became_active":"2024-03-29T04:07:41.424814+0000","last_became_peered":"2024-03-29T04:07:41.424814+0000","last_unstale":"2024-03-29T04:07:42.798579+0000","last_undegraded":"2024-03-29T04:07:42.798579+0000","last_fullsized":"2024-03-29T04:07:42.798579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:40.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797416+0000","last_change":"2024-03-29T04:07:40.429808+0000","last_active":"2024-03-29T04:07:42.797416+0000","last_peered":"2024-03-29T04:07:42.797416+0000","last_clean":"2024-03-29T04:07:42.797416+0000","last_became_active":"2024-03-29T04:07:40.428865+0000","last_became_peered":"2024-03-29T04:07:40.428865+0000","last_unstale":"2024-03-29T04:07:42.797416+0000","last_undegraded":"2024-03-29T04:07:42.797416+0000","last_fullsized":"2024-03-29T04:07:42.797416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:44.678749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417729+0000","last_change":"2024-03-29T04:07:41.434698+0000","last_active":"2024-03-29T04:07:42.417729+0000","last_peered":"2024-03-29T04:07:42.417729+0000","last_clean":"2024-03-29T04:07:42.417729+0000","last_became_active":"2024-03-29T04:07:41.434270+0000","last_became_peered":"2024-03-29T04:07:41.434270+0000","last_unstale":"2024-03-29T04:07:42.417729+0000","last_undegraded":"2024-03-29T04:07:42.417729+0000","last_fullsized":"2024-03-29T04:07:42.417729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:42.838588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413481+0000","last_change":"2024-03-29T04:07:40.425280+0000","last_active":"2024-03-29T04:07:42.413481+0000","last_peered":"2024-03-29T04:07:42.413481+0000","last_clean":"2024-03-29T04:07:42.413481+0000","last_became_active":"2024-03-29T04:07:40.424998+0000","last_became_peered":"2024-03-29T04:07:40.424998+0000","last_unstale":"2024-03-29T04:07:42.413481+0000","last_undegraded":"2024-03-29T04:07:42.413481+0000","last_fullsized":"2024-03-29T04:07:42.413481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:34.960075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798861+0000","last_change":"2024-03-29T04:07:41.424267+0000","last_active":"2024-03-29T04:07:42.798861+0000","last_peered":"2024-03-29T04:07:42.798861+0000","last_clean":"2024-03-29T04:07:42.798861+0000","last_became_active":"2024-03-29T04:07:41.423920+0000","last_became_peered":"2024-03-29T04:07:41.423920+0000","last_unstale":"2024-03-29T04:07:42.798861+0000","last_undegraded":"2024-03-29T04:07:42.798861+0000","last_fullsized":"2024-03-29T04:07:42.798861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:32:16.919913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413416+0000","last_change":"2024-03-29T04:07:40.422602+0000","last_active":"2024-03-29T04:07:42.413416+0000","last_peered":"2024-03-29T04:07:42.413416+0000","last_clean":"2024-03-29T04:07:42.413416+0000","last_became_active":"2024-03-29T04:07:40.422332+0000","last_became_peered":"2024-03-29T04:07:40.422332+0000","last_unstale":"2024-03-29T04:07:42.413416+0000","last_undegraded":"2024-03-29T04:07:42.413416+0000","last_fullsized":"2024-03-29T04:07:42.413416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.799042+0000","last_change":"2024-03-29T04:07:41.427166+0000","last_active":"2024-03-29T04:07:42.799042+0000","last_peered":"2024-03-29T04:07:42.799042+0000","last_clean":"2024-03-29T04:07:42.799042+0000","last_became_active":"2024-03-29T04:07:41.426914+0000","last_became_peered":"2024-03-29T04:07:41.426914+0000","last_unstale":"2024-03-29T04:07:42.799042+0000","last_undegraded":"2024-03-29T04:07:42.799042+0000","last_fullsized":"2024-03-29T04:07:42.799042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:58.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798369+0000","last_change":"2024-03-29T04:07:40.430896+0000","last_active":"2024-03-29T04:07:42.798369+0000","last_peered":"2024-03-29T04:07:42.798369+0000","last_clean":"2024-03-29T04:07:42.798369+0000","last_became_active":"2024-03-29T04:07:40.430549+0000","last_became_peered":"2024-03-29T04:07:40.430549+0000","last_unstale":"2024-03-29T04:07:42.798369+0000","last_undegraded":"2024-03-29T04:07:42.798369+0000","last_fullsized":"2024-03-29T04:07:42.798369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:39:46.902638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417189+0000","last_change":"2024-03-29T04:07:41.429537+0000","last_active":"2024-03-29T04:07:42.417189+0000","last_peered":"2024-03-29T04:07:42.417189+0000","last_clean":"2024-03-29T04:07:42.417189+0000","last_became_active":"2024-03-29T04:07:41.429173+0000","last_became_peered":"2024-03-29T04:07:41.429173+0000","last_unstale":"2024-03-29T04:07:42.417189+0000","last_undegraded":"2024-03-29T04:07:42.417189+0000","last_fullsized":"2024-03-29T04:07:42.417189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:57.438333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798597+0000","last_change":"2024-03-29T04:07:41.436387+0000","last_active":"2024-03-29T04:07:42.798597+0000","last_peered":"2024-03-29T04:07:42.798597+0000","last_clean":"2024-03-29T04:07:42.798597+0000","last_became_active":"2024-03-29T04:07:41.436043+0000","last_became_peered":"2024-03-29T04:07:41.436043+0000","last_unstale":"2024-03-29T04:07:42.798597+0000","last_undegraded":"2024-03-29T04:07:42.798597+0000","last_fullsized":"2024-03-29T04:07:42.798597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:12:50.176893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798007+0000","last_change":"2024-03-29T04:07:40.431266+0000","last_active":"2024-03-29T04:07:42.798007+0000","last_peered":"2024-03-29T04:07:42.798007+0000","last_clean":"2024-03-29T04:07:42.798007+0000","last_became_active":"2024-03-29T04:07:40.430984+0000","last_became_peered":"2024-03-29T04:07:40.430984+0000","last_unstale":"2024-03-29T04:07:42.798007+0000","last_undegraded":"2024-03-29T04:07:42.798007+0000","last_fullsized":"2024-03-29T04:07:42.798007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:41.833725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798023+0000","last_change":"2024-03-29T04:07:41.436711+0000","last_active":"2024-03-29T04:07:42.798023+0000","last_peered":"2024-03-29T04:07:42.798023+0000","last_clean":"2024-03-29T04:07:42.798023+0000","last_became_active":"2024-03-29T04:07:41.436145+0000","last_became_peered":"2024-03-29T04:07:41.436145+0000","last_unstale":"2024-03-29T04:07:42.798023+0000","last_undegraded":"2024-03-29T04:07:42.798023+0000","last_fullsized":"2024-03-29T04:07:42.798023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:58.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798296+0000","last_change":"2024-03-29T04:07:40.430774+0000","last_active":"2024-03-29T04:07:42.798296+0000","last_peered":"2024-03-29T04:07:42.798296+0000","last_clean":"2024-03-29T04:07:42.798296+0000","last_became_active":"2024-03-29T04:07:40.430503+0000","last_became_peered":"2024-03-29T04:07:40.430503+0000","last_unstale":"2024-03-29T04:07:42.798296+0000","last_undegraded":"2024-03-29T04:07:42.798296+0000","last_fullsized":"2024-03-29T04:07:42.798296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:37.084723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797696+0000","last_change":"2024-03-29T04:07:41.441323+0000","last_active":"2024-03-29T04:07:42.797696+0000","last_peered":"2024-03-29T04:07:42.797696+0000","last_clean":"2024-03-29T04:07:42.797696+0000","last_became_active":"2024-03-29T04:07:41.441074+0000","last_became_peered":"2024-03-29T04:07:41.441074+0000","last_unstale":"2024-03-29T04:07:42.797696+0000","last_undegraded":"2024-03-29T04:07:42.797696+0000","last_fullsized":"2024-03-29T04:07:42.797696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:36:03.632489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798948+0000","last_change":"2024-03-29T04:07:40.431979+0000","last_active":"2024-03-29T04:07:42.798948+0000","last_peered":"2024-03-29T04:07:42.798948+0000","last_clean":"2024-03-29T04:07:42.798948+0000","last_became_active":"2024-03-29T04:07:40.431702+0000","last_became_peered":"2024-03-29T04:07:40.431702+0000","last_unstale":"2024-03-29T04:07:42.798948+0000","last_undegraded":"2024-03-29T04:07:42.798948+0000","last_fullsized":"2024-03-29T04:07:42.798948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798867+0000","last_change":"2024-03-29T04:07:41.427017+0000","last_active":"2024-03-29T04:07:42.798867+0000","last_peered":"2024-03-29T04:07:42.798867+0000","last_clean":"2024-03-29T04:07:42.798867+0000","last_became_active":"2024-03-29T04:07:41.426749+0000","last_became_peered":"2024-03-29T04:07:41.426749+0000","last_unstale":"2024-03-29T04:07:42.798867+0000","last_undegraded":"2024-03-29T04:07:42.798867+0000","last_fullsized":"2024-03-29T04:07:42.798867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:27:37.837537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797804+0000","last_change":"2024-03-29T04:07:40.421536+0000","last_active":"2024-03-29T04:07:42.797804+0000","last_peered":"2024-03-29T04:07:42.797804+0000","last_clean":"2024-03-29T04:07:42.797804+0000","last_became_active":"2024-03-29T04:07:40.421279+0000","last_became_peered":"2024-03-29T04:07:40.421279+0000","last_unstale":"2024-03-29T04:07:42.797804+0000","last_undegraded":"2024-03-29T04:07:42.797804+0000","last_fullsized":"2024-03-29T04:07:42.797804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:43.704154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:07:48.720 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:07:48.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.877+0000 7f132f160640 1 Processor -- start 2024-03-29T04:07:48.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.877+0000 7f132f160640 1 -- start start 2024-03-29T04:07:48.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f132f160640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f132806e1c0 0x7f132806e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f132f160640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13280b3c10 con 0x7f1328144790 2024-03-29T04:07:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f132f160640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13280b3ef0 con 0x7f132806f1e0 2024-03-29T04:07:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f132f160640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f13280b39d0 con 0x7f132806e1c0 2024-03-29T04:07:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f132cefc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1328144790 0x7f13280b4cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50486/0 (socket says 172.21.15.97:50486) 2024-03-29T04:07:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f132cefc640 1 -- 172.21.15.97:0/680562871 learned_addr learned my addr 172.21.15.97:0/680562871 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f132d6fd640 1 --2- 172.21.15.97:0/680562871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f132806e1c0 0x7f132806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f13277fe640 1 -- 172.21.15.97:0/680562871 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3687108556 0 0) 0x7f13280b3c10 con 0x7f1328144790 2024-03-29T04:07:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f13277fe640 1 -- 172.21.15.97:0/680562871 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1318004630 con 0x7f1328144790 2024-03-29T04:07:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f13277fe640 1 -- 172.21.15.97:0/680562871 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3957726723 0 0) 0x7f13280b3ef0 con 0x7f132806f1e0 2024-03-29T04:07:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f13277fe640 1 -- 172.21.15.97:0/680562871 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13280b3c10 con 0x7f132806f1e0 2024-03-29T04:07:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f132d6fd640 1 -- 172.21.15.97:0/680562871 >> v1:172.21.15.154:6789/0 conn(0x7f132806f1e0 legacy=0x7f13280b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f132d6fd640 1 -- 172.21.15.97:0/680562871 >> v1:172.21.15.97:6789/0 conn(0x7f1328144790 legacy=0x7f13280b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.878+0000 7f132d6fd640 1 -- 172.21.15.97:0/680562871 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13280b41d0 con 0x7f132806e1c0 2024-03-29T04:07:48.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.879+0000 7f132d6fd640 1 --2- 172.21.15.97:0/680562871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f132806e1c0 0x7f132806e670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1320009dc0 tx=0x7f132002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d1f6d0e7fe45137 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:48.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.879+0000 7f13277fe640 1 -- 172.21.15.97:0/680562871 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320039070 con 0x7f132806e1c0 2024-03-29T04:07:48.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.879+0000 7f13277fe640 1 -- 172.21.15.97:0/680562871 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132002ccd0 con 0x7f132806e1c0 2024-03-29T04:07:48.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.879+0000 7f13277fe640 1 -- 172.21.15.97:0/680562871 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320034640 con 0x7f132806e1c0 2024-03-29T04:07:48.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.880+0000 7f132f160640 1 -- 172.21.15.97:0/680562871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f132806e1c0 msgr2=0x7f132806e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.880+0000 7f132f160640 1 --2- 172.21.15.97:0/680562871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f132806e1c0 0x7f132806e670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1320009dc0 tx=0x7f132002bf20 comp rx=0 tx=0).stop 2024-03-29T04:07:48.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.880+0000 7f132f160640 1 -- 172.21.15.97:0/680562871 shutdown_connections 2024-03-29T04:07:48.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.880+0000 7f132f160640 1 --2- 172.21.15.97:0/680562871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f132806e1c0 0x7f132806e670 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.880+0000 7f132f160640 1 -- 172.21.15.97:0/680562871 >> 172.21.15.97:0/680562871 conn(0x7f132806b510 msgr2=0x7f132806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:48.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.880+0000 7f132f160640 1 -- 172.21.15.97:0/680562871 shutdown_connections 2024-03-29T04:07:48.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.880+0000 7f132f160640 1 -- 172.21.15.97:0/680562871 wait complete. 2024-03-29T04:07:48.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.881+0000 7f132f160640 1 Processor -- start 2024-03-29T04:07:48.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.881+0000 7f132f160640 1 -- start start 2024-03-29T04:07:48.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132f160640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132806e1c0 0x7f13281f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:48.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132f160640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f132806f1e0 0x7f13280cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:48.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132f160640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328144790 0x7f13280cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:48.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132f160640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f132806d930 con 0x7f1328144790 2024-03-29T04:07:48.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132f160640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f13280b4790 con 0x7f132806e1c0 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132f160640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f132806dab0 con 0x7f132806f1e0 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132d6fd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328144790 0x7f13280cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132d6fd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328144790 0x7f13280cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39150/0 (socket says 172.21.15.97:39150) 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132d6fd640 1 -- 172.21.15.97:0/320223898 learned_addr learned my addr 172.21.15.97:0/320223898 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132cefc640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132806e1c0 0x7f13281f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f1327fff640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f132806f1e0 0x7f13280cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132d6fd640 1 -- 172.21.15.97:0/320223898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f132806f1e0 msgr2=0x7f13280cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132d6fd640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f132806f1e0 0x7f13280cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132d6fd640 1 -- 172.21.15.97:0/320223898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132806e1c0 msgr2=0x7f13281f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132d6fd640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132806e1c0 0x7f13281f41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132d6fd640 1 -- 172.21.15.97:0/320223898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13280cfad0 con 0x7f1328144790 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132d6fd640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328144790 0x7f13280cdbd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1320009ee0 tx=0x7f1320004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f13257fa640 1 -- 172.21.15.97:0/320223898 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320039050 con 0x7f1328144790 2024-03-29T04:07:48.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f13257fa640 1 -- 172.21.15.97:0/320223898 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1320045070 con 0x7f1328144790 2024-03-29T04:07:48.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f13257fa640 1 -- 172.21.15.97:0/320223898 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320009450 con 0x7f1328144790 2024-03-29T04:07:48.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132f160640 1 -- 172.21.15.97:0/320223898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13280b0720 con 0x7f1328144790 2024-03-29T04:07:48.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.882+0000 7f132f160640 1 -- 172.21.15.97:0/320223898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13281f1490 con 0x7f1328144790 2024-03-29T04:07:48.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.883+0000 7f132f160640 1 -- 172.21.15.97:0/320223898 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f132806fc00 con 0x7f1328144790 2024-03-29T04:07:48.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.884+0000 7f13257fa640 1 -- 172.21.15.97:0/320223898 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f132003e850 con 0x7f1328144790 2024-03-29T04:07:48.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.884+0000 7f13257fa640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f130c06c530 0x7f130c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:48.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.884+0000 7f13257fa640 1 -- 172.21.15.97:0/320223898 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f13200ae380 con 0x7f1328144790 2024-03-29T04:07:48.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.884+0000 7f132cefc640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f130c06c530 0x7f130c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:48.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.885+0000 7f132cefc640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f130c06c530 0x7f130c06e9f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f131c000f60 tx=0x7f131c007d70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:48.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:48.889+0000 7f13257fa640 1 -- 172.21.15.97:0/320223898 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f132806fc00 con 0x7f1328144790 2024-03-29T04:07:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.058+0000 7f132f160640 1 -- 172.21.15.97:0/320223898 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f13280b3de0 con 0x7f130c06c530 2024-03-29T04:07:49.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.064+0000 7f13257fa640 1 -- 172.21.15.97:0/320223898 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f13280b3de0 con 0x7f130c06c530 2024-03-29T04:07:49.047 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:49.048 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-29T04:07:49.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.067+0000 7f132f160640 1 -- 172.21.15.97:0/320223898 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f130c06c530 msgr2=0x7f130c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.067+0000 7f132f160640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f130c06c530 0x7f130c06e9f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f131c000f60 tx=0x7f131c007d70 comp rx=0 tx=0).stop 2024-03-29T04:07:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.067+0000 7f132f160640 1 -- 172.21.15.97:0/320223898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328144790 msgr2=0x7f13280cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.067+0000 7f132f160640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328144790 0x7f13280cdbd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1320009ee0 tx=0x7f1320004a40 comp rx=0 tx=0).stop 2024-03-29T04:07:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.067+0000 7f132f160640 1 -- 172.21.15.97:0/320223898 shutdown_connections 2024-03-29T04:07:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.067+0000 7f132f160640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f130c06c530 0x7f130c06e9f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.067+0000 7f132f160640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328144790 0x7f13280cdbd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.067+0000 7f132f160640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f132806f1e0 0x7f13280cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.067+0000 7f132f160640 1 --2- 172.21.15.97:0/320223898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132806e1c0 0x7f13281f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.067+0000 7f132f160640 1 -- 172.21.15.97:0/320223898 >> 172.21.15.97:0/320223898 conn(0x7f132806b510 msgr2=0x7f13280b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.067+0000 7f132f160640 1 -- 172.21.15.97:0/320223898 shutdown_connections 2024-03-29T04:07:49.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.067+0000 7f132f160640 1 -- 172.21.15.97:0/320223898 wait complete. 2024-03-29T04:07:49.058 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-29T04:07:48.640081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001576"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798559+0000","last_change":"2024-03-29T04:07:41.422348+0000","last_active":"2024-03-29T04:07:42.798559+0000","last_peered":"2024-03-29T04:07:42.798559+0000","last_clean":"2024-03-29T04:07:42.798559+0000","last_became_active":"2024-03-29T04:07:41.422010+0000","last_became_peered":"2024-03-29T04:07:41.422010+0000","last_unstale":"2024-03-29T04:07:42.798559+0000","last_undegraded":"2024-03-29T04:07:42.798559+0000","last_fullsized":"2024-03-29T04:07:42.798559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:45:40.024022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413460+0000","last_change":"2024-03-29T04:07:41.426583+0000","last_active":"2024-03-29T04:07:42.413460+0000","last_peered":"2024-03-29T04:07:42.413460+0000","last_clean":"2024-03-29T04:07:42.413460+0000","last_became_active":"2024-03-29T04:07:41.426321+0000","last_became_peered":"2024-03-29T04:07:41.426321+0000","last_unstale":"2024-03-29T04:07:42.413460+0000","last_undegraded":"2024-03-29T04:07:42.413460+0000","last_fullsized":"2024-03-29T04:07:42.413460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:28:53.046219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798410+0000","last_change":"2024-03-29T04:07:41.430842+0000","last_active":"2024-03-29T04:07:42.798410+0000","last_peered":"2024-03-29T04:07:42.798410+0000","last_clean":"2024-03-29T04:07:42.798410+0000","last_became_active":"2024-03-29T04:07:41.430402+0000","last_became_peered":"2024-03-29T04:07:41.430402+0000","last_unstale":"2024-03-29T04:07:42.798410+0000","last_undegraded":"2024-03-29T04:07:42.798410+0000","last_fullsized":"2024-03-29T04:07:42.798410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:23:32.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798165+0000","last_change":"2024-03-29T04:07:41.436930+0000","last_active":"2024-03-29T04:07:42.798165+0000","last_peered":"2024-03-29T04:07:42.798165+0000","last_clean":"2024-03-29T04:07:42.798165+0000","last_became_active":"2024-03-29T04:07:41.436434+0000","last_became_peered":"2024-03-29T04:07:41.436434+0000","last_unstale":"2024-03-29T04:07:42.798165+0000","last_undegraded":"2024-03-29T04:07:42.798165+0000","last_fullsized":"2024-03-29T04:07:42.798165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:53:03.772026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798028+0000","last_change":"2024-03-29T04:07:41.428211+0000","last_active":"2024-03-29T04:07:42.798028+0000","last_peered":"2024-03-29T04:07:42.798028+0000","last_clean":"2024-03-29T04:07:42.798028+0000","last_became_active":"2024-03-29T04:07:41.427906+0000","last_became_peered":"2024-03-29T04:07:41.427906+0000","last_unstale":"2024-03-29T04:07:42.798028+0000","last_undegraded":"2024-03-29T04:07:42.798028+0000","last_fullsized":"2024-03-29T04:07:42.798028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:27:35.869429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797533+0000","last_change":"2024-03-29T04:07:41.438804+0000","last_active":"2024-03-29T04:07:42.797533+0000","last_peered":"2024-03-29T04:07:42.797533+0000","last_clean":"2024-03-29T04:07:42.797533+0000","last_became_active":"2024-03-29T04:07:41.438533+0000","last_became_peered":"2024-03-29T04:07:41.438533+0000","last_unstale":"2024-03-29T04:07:42.797533+0000","last_undegraded":"2024-03-29T04:07:42.797533+0000","last_fullsized":"2024-03-29T04:07:42.797533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:08.125575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797742+0000","last_change":"2024-03-29T04:07:41.426635+0000","last_active":"2024-03-29T04:07:42.797742+0000","last_peered":"2024-03-29T04:07:42.797742+0000","last_clean":"2024-03-29T04:07:42.797742+0000","last_became_active":"2024-03-29T04:07:41.426301+0000","last_became_peered":"2024-03-29T04:07:41.426301+0000","last_unstale":"2024-03-29T04:07:42.797742+0000","last_undegraded":"2024-03-29T04:07:42.797742+0000","last_fullsized":"2024-03-29T04:07:42.797742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:27:37.837537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798679+0000","last_change":"2024-03-29T04:07:41.436075+0000","last_active":"2024-03-29T04:07:42.798679+0000","last_peered":"2024-03-29T04:07:42.798679+0000","last_clean":"2024-03-29T04:07:42.798679+0000","last_became_active":"2024-03-29T04:07:41.435833+0000","last_became_peered":"2024-03-29T04:07:41.435833+0000","last_unstale":"2024-03-29T04:07:42.798679+0000","last_undegraded":"2024-03-29T04:07:42.798679+0000","last_fullsized":"2024-03-29T04:07:42.798679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:23:32.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798649+0000","last_change":"2024-03-29T04:07:41.441371+0000","last_active":"2024-03-29T04:07:42.798649+0000","last_peered":"2024-03-29T04:07:42.798649+0000","last_clean":"2024-03-29T04:07:42.798649+0000","last_became_active":"2024-03-29T04:07:41.441008+0000","last_became_peered":"2024-03-29T04:07:41.441008+0000","last_unstale":"2024-03-29T04:07:42.798649+0000","last_undegraded":"2024-03-29T04:07:42.798649+0000","last_fullsized":"2024-03-29T04:07:42.798649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:14:40.086642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417268+0000","last_change":"2024-03-29T04:07:41.436960+0000","last_active":"2024-03-29T04:07:42.417268+0000","last_peered":"2024-03-29T04:07:42.417268+0000","last_clean":"2024-03-29T04:07:42.417268+0000","last_became_active":"2024-03-29T04:07:41.436652+0000","last_became_peered":"2024-03-29T04:07:41.436652+0000","last_unstale":"2024-03-29T04:07:42.417268+0000","last_undegraded":"2024-03-29T04:07:42.417268+0000","last_fullsized":"2024-03-29T04:07:42.417268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:23:32.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417196+0000","last_change":"2024-03-29T04:07:41.437571+0000","last_active":"2024-03-29T04:07:42.417196+0000","last_peered":"2024-03-29T04:07:42.417196+0000","last_clean":"2024-03-29T04:07:42.417196+0000","last_became_active":"2024-03-29T04:07:41.437307+0000","last_became_peered":"2024-03-29T04:07:41.437307+0000","last_unstale":"2024-03-29T04:07:42.417196+0000","last_undegraded":"2024-03-29T04:07:42.417196+0000","last_fullsized":"2024-03-29T04:07:42.417196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:30:59.143986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417226+0000","last_change":"2024-03-29T04:07:41.429087+0000","last_active":"2024-03-29T04:07:42.417226+0000","last_peered":"2024-03-29T04:07:42.417226+0000","last_clean":"2024-03-29T04:07:42.417226+0000","last_became_active":"2024-03-29T04:07:41.427764+0000","last_became_peered":"2024-03-29T04:07:41.427764+0000","last_unstale":"2024-03-29T04:07:42.417226+0000","last_undegraded":"2024-03-29T04:07:42.417226+0000","last_fullsized":"2024-03-29T04:07:42.417226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:12.561151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.799039+0000","last_change":"2024-03-29T04:07:41.427842+0000","last_active":"2024-03-29T04:07:42.799039+0000","last_peered":"2024-03-29T04:07:42.799039+0000","last_clean":"2024-03-29T04:07:42.799039+0000","last_became_active":"2024-03-29T04:07:41.427481+0000","last_became_peered":"2024-03-29T04:07:41.427481+0000","last_unstale":"2024-03-29T04:07:42.799039+0000","last_undegraded":"2024-03-29T04:07:42.799039+0000","last_fullsized":"2024-03-29T04:07:42.799039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:53:03.772026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798076+0000","last_change":"2024-03-29T04:07:41.425389+0000","last_active":"2024-03-29T04:07:42.798076+0000","last_peered":"2024-03-29T04:07:42.798076+0000","last_clean":"2024-03-29T04:07:42.798076+0000","last_became_active":"2024-03-29T04:07:41.425016+0000","last_became_peered":"2024-03-29T04:07:41.425016+0000","last_unstale":"2024-03-29T04:07:42.798076+0000","last_undegraded":"2024-03-29T04:07:42.798076+0000","last_fullsized":"2024-03-29T04:07:42.798076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:57.438333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413581+0000","last_change":"2024-03-29T04:07:41.428282+0000","last_active":"2024-03-29T04:07:42.413581+0000","last_peered":"2024-03-29T04:07:42.413581+0000","last_clean":"2024-03-29T04:07:42.413581+0000","last_became_active":"2024-03-29T04:07:41.428042+0000","last_became_peered":"2024-03-29T04:07:41.428042+0000","last_unstale":"2024-03-29T04:07:42.413581+0000","last_undegraded":"2024-03-29T04:07:42.413581+0000","last_fullsized":"2024-03-29T04:07:42.413581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:23:32.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797464+0000","last_change":"2024-03-29T04:07:41.434962+0000","last_active":"2024-03-29T04:07:42.797464+0000","last_peered":"2024-03-29T04:07:42.797464+0000","last_clean":"2024-03-29T04:07:42.797464+0000","last_became_active":"2024-03-29T04:07:41.434691+0000","last_became_peered":"2024-03-29T04:07:41.434691+0000","last_unstale":"2024-03-29T04:07:42.797464+0000","last_undegraded":"2024-03-29T04:07:42.797464+0000","last_fullsized":"2024-03-29T04:07:42.797464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:23:32.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798581+0000","last_change":"2024-03-29T04:07:41.434716+0000","last_active":"2024-03-29T04:07:42.798581+0000","last_peered":"2024-03-29T04:07:42.798581+0000","last_clean":"2024-03-29T04:07:42.798581+0000","last_became_active":"2024-03-29T04:07:41.434372+0000","last_became_peered":"2024-03-29T04:07:41.434372+0000","last_unstale":"2024-03-29T04:07:42.798581+0000","last_undegraded":"2024-03-29T04:07:42.798581+0000","last_fullsized":"2024-03-29T04:07:42.798581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:14.560489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797999+0000","last_change":"2024-03-29T04:07:41.427613+0000","last_active":"2024-03-29T04:07:42.797999+0000","last_peered":"2024-03-29T04:07:42.797999+0000","last_clean":"2024-03-29T04:07:42.797999+0000","last_became_active":"2024-03-29T04:07:41.427338+0000","last_became_peered":"2024-03-29T04:07:41.427338+0000","last_unstale":"2024-03-29T04:07:42.797999+0000","last_undegraded":"2024-03-29T04:07:42.797999+0000","last_fullsized":"2024-03-29T04:07:42.797999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:42.838588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798413+0000","last_change":"2024-03-29T04:07:41.427942+0000","last_active":"2024-03-29T04:07:42.798413+0000","last_peered":"2024-03-29T04:07:42.798413+0000","last_clean":"2024-03-29T04:07:42.798413+0000","last_became_active":"2024-03-29T04:07:41.427667+0000","last_became_peered":"2024-03-29T04:07:41.427667+0000","last_unstale":"2024-03-29T04:07:42.798413+0000","last_undegraded":"2024-03-29T04:07:42.798413+0000","last_fullsized":"2024-03-29T04:07:42.798413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:12:50.176893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417255+0000","last_change":"2024-03-29T04:07:41.429646+0000","last_active":"2024-03-29T04:07:42.417255+0000","last_peered":"2024-03-29T04:07:42.417255+0000","last_clean":"2024-03-29T04:07:42.417255+0000","last_became_active":"2024-03-29T04:07:41.429376+0000","last_became_peered":"2024-03-29T04:07:41.429376+0000","last_unstale":"2024-03-29T04:07:42.417255+0000","last_undegraded":"2024-03-29T04:07:42.417255+0000","last_fullsized":"2024-03-29T04:07:42.417255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:35.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797955+0000","last_change":"2024-03-29T04:07:41.425314+0000","last_active":"2024-03-29T04:07:42.797955+0000","last_peered":"2024-03-29T04:07:42.797955+0000","last_clean":"2024-03-29T04:07:42.797955+0000","last_became_active":"2024-03-29T04:07:41.424897+0000","last_became_peered":"2024-03-29T04:07:41.424897+0000","last_unstale":"2024-03-29T04:07:42.797955+0000","last_undegraded":"2024-03-29T04:07:42.797955+0000","last_fullsized":"2024-03-29T04:07:42.797955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:31:27.159107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797567+0000","last_change":"2024-03-29T04:07:41.441556+0000","last_active":"2024-03-29T04:07:42.797567+0000","last_peered":"2024-03-29T04:07:42.797567+0000","last_clean":"2024-03-29T04:07:42.797567+0000","last_became_active":"2024-03-29T04:07:41.437251+0000","last_became_peered":"2024-03-29T04:07:41.437251+0000","last_unstale":"2024-03-29T04:07:42.797567+0000","last_undegraded":"2024-03-29T04:07:42.797567+0000","last_fullsized":"2024-03-29T04:07:42.797567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:32:16.919913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798200+0000","last_change":"2024-03-29T04:07:41.435226+0000","last_active":"2024-03-29T04:07:42.798200+0000","last_peered":"2024-03-29T04:07:42.798200+0000","last_clean":"2024-03-29T04:07:42.798200+0000","last_became_active":"2024-03-29T04:07:41.434543+0000","last_became_peered":"2024-03-29T04:07:41.434543+0000","last_unstale":"2024-03-29T04:07:42.798200+0000","last_undegraded":"2024-03-29T04:07:42.798200+0000","last_fullsized":"2024-03-29T04:07:42.798200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:30:59.143986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797501+0000","last_change":"2024-03-29T04:07:41.434893+0000","last_active":"2024-03-29T04:07:42.797501+0000","last_peered":"2024-03-29T04:07:42.797501+0000","last_clean":"2024-03-29T04:07:42.797501+0000","last_became_active":"2024-03-29T04:07:41.434578+0000","last_became_peered":"2024-03-29T04:07:41.434578+0000","last_unstale":"2024-03-29T04:07:42.797501+0000","last_undegraded":"2024-03-29T04:07:42.797501+0000","last_fullsized":"2024-03-29T04:07:42.797501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:40.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798261+0000","last_change":"2024-03-29T04:07:41.434765+0000","last_active":"2024-03-29T04:07:42.798261+0000","last_peered":"2024-03-29T04:07:42.798261+0000","last_clean":"2024-03-29T04:07:42.798261+0000","last_became_active":"2024-03-29T04:07:41.434463+0000","last_became_peered":"2024-03-29T04:07:41.434463+0000","last_unstale":"2024-03-29T04:07:42.798261+0000","last_undegraded":"2024-03-29T04:07:42.798261+0000","last_fullsized":"2024-03-29T04:07:42.798261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:28:53.046219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417218+0000","last_change":"2024-03-29T04:07:41.424421+0000","last_active":"2024-03-29T04:07:42.417218+0000","last_peered":"2024-03-29T04:07:42.417218+0000","last_clean":"2024-03-29T04:07:42.417218+0000","last_became_active":"2024-03-29T04:07:41.424154+0000","last_became_peered":"2024-03-29T04:07:41.424154+0000","last_unstale":"2024-03-29T04:07:42.417218+0000","last_undegraded":"2024-03-29T04:07:42.417218+0000","last_fullsized":"2024-03-29T04:07:42.417218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:28:53.046219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413456+0000","last_change":"2024-03-29T04:07:41.429051+0000","last_active":"2024-03-29T04:07:42.413456+0000","last_peered":"2024-03-29T04:07:42.413456+0000","last_clean":"2024-03-29T04:07:42.413456+0000","last_became_active":"2024-03-29T04:07:41.428748+0000","last_became_peered":"2024-03-29T04:07:41.428748+0000","last_unstale":"2024-03-29T04:07:42.413456+0000","last_undegraded":"2024-03-29T04:07:42.413456+0000","last_fullsized":"2024-03-29T04:07:42.413456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:30:59.143986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798736+0000","last_change":"2024-03-29T04:07:41.435463+0000","last_active":"2024-03-29T04:07:42.798736+0000","last_peered":"2024-03-29T04:07:42.798736+0000","last_clean":"2024-03-29T04:07:42.798736+0000","last_became_active":"2024-03-29T04:07:41.434902+0000","last_became_peered":"2024-03-29T04:07:41.434902+0000","last_unstale":"2024-03-29T04:07:42.798736+0000","last_undegraded":"2024-03-29T04:07:42.798736+0000","last_fullsized":"2024-03-29T04:07:42.798736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:40.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798097+0000","last_change":"2024-03-29T04:07:41.428583+0000","last_active":"2024-03-29T04:07:42.798097+0000","last_peered":"2024-03-29T04:07:42.798097+0000","last_clean":"2024-03-29T04:07:42.798097+0000","last_became_active":"2024-03-29T04:07:41.428264+0000","last_became_peered":"2024-03-29T04:07:41.428264+0000","last_unstale":"2024-03-29T04:07:42.798097+0000","last_undegraded":"2024-03-29T04:07:42.798097+0000","last_fullsized":"2024-03-29T04:07:42.798097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:30:59.143986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797685+0000","last_change":"2024-03-29T04:07:41.433555+0000","last_active":"2024-03-29T04:07:42.797685+0000","last_peered":"2024-03-29T04:07:42.797685+0000","last_clean":"2024-03-29T04:07:42.797685+0000","last_became_active":"2024-03-29T04:07:41.433253+0000","last_became_peered":"2024-03-29T04:07:41.433253+0000","last_unstale":"2024-03-29T04:07:42.797685+0000","last_undegraded":"2024-03-29T04:07:42.797685+0000","last_fullsized":"2024-03-29T04:07:42.797685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:27:37.837537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797593+0000","last_change":"2024-03-29T04:07:41.433903+0000","last_active":"2024-03-29T04:07:42.797593+0000","last_peered":"2024-03-29T04:07:42.797593+0000","last_clean":"2024-03-29T04:07:42.797593+0000","last_became_active":"2024-03-29T04:07:41.433582+0000","last_became_peered":"2024-03-29T04:07:41.433582+0000","last_unstale":"2024-03-29T04:07:42.797593+0000","last_undegraded":"2024-03-29T04:07:42.797593+0000","last_fullsized":"2024-03-29T04:07:42.797593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:45.683611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417165+0000","last_change":"2024-03-29T04:07:41.426138+0000","last_active":"2024-03-29T04:07:42.417165+0000","last_peered":"2024-03-29T04:07:42.417165+0000","last_clean":"2024-03-29T04:07:42.417165+0000","last_became_active":"2024-03-29T04:07:41.425636+0000","last_became_peered":"2024-03-29T04:07:41.425636+0000","last_unstale":"2024-03-29T04:07:42.417165+0000","last_undegraded":"2024-03-29T04:07:42.417165+0000","last_fullsized":"2024-03-29T04:07:42.417165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:31:27.159107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417281+0000","last_change":"2024-03-29T04:07:41.434449+0000","last_active":"2024-03-29T04:07:42.417281+0000","last_peered":"2024-03-29T04:07:42.417281+0000","last_clean":"2024-03-29T04:07:42.417281+0000","last_became_active":"2024-03-29T04:07:41.434169+0000","last_became_peered":"2024-03-29T04:07:41.434169+0000","last_unstale":"2024-03-29T04:07:42.417281+0000","last_undegraded":"2024-03-29T04:07:42.417281+0000","last_fullsized":"2024-03-29T04:07:42.417281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:35.964938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.799089+0000","last_change":"2024-03-29T04:07:40.425200+0000","last_active":"2024-03-29T04:07:42.799089+0000","last_peered":"2024-03-29T04:07:42.799089+0000","last_clean":"2024-03-29T04:07:42.799089+0000","last_became_active":"2024-03-29T04:07:40.424061+0000","last_became_peered":"2024-03-29T04:07:40.424061+0000","last_unstale":"2024-03-29T04:07:42.799089+0000","last_undegraded":"2024-03-29T04:07:42.799089+0000","last_fullsized":"2024-03-29T04:07:42.799089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:04.924835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798235+0000","last_change":"2024-03-29T04:07:41.432958+0000","last_active":"2024-03-29T04:07:42.798235+0000","last_peered":"2024-03-29T04:07:42.798235+0000","last_clean":"2024-03-29T04:07:42.798235+0000","last_became_active":"2024-03-29T04:07:41.432654+0000","last_became_peered":"2024-03-29T04:07:41.432654+0000","last_unstale":"2024-03-29T04:07:42.798235+0000","last_undegraded":"2024-03-29T04:07:42.798235+0000","last_fullsized":"2024-03-29T04:07:42.798235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:35.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.416646+0000","last_change":"2024-03-29T04:07:40.418823+0000","last_active":"2024-03-29T04:07:42.416646+0000","last_peered":"2024-03-29T04:07:42.416646+0000","last_clean":"2024-03-29T04:07:42.416646+0000","last_became_active":"2024-03-29T04:07:40.418542+0000","last_became_peered":"2024-03-29T04:07:40.418542+0000","last_unstale":"2024-03-29T04:07:42.416646+0000","last_undegraded":"2024-03-29T04:07:42.416646+0000","last_fullsized":"2024-03-29T04:07:42.416646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:04.924835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798353+0000","last_change":"2024-03-29T04:07:40.430532+0000","last_active":"2024-03-29T04:07:42.798353+0000","last_peered":"2024-03-29T04:07:42.798353+0000","last_clean":"2024-03-29T04:07:42.798353+0000","last_became_active":"2024-03-29T04:07:40.430321+0000","last_became_peered":"2024-03-29T04:07:40.430321+0000","last_unstale":"2024-03-29T04:07:42.798353+0000","last_undegraded":"2024-03-29T04:07:42.798353+0000","last_fullsized":"2024-03-29T04:07:42.798353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:40:12.105363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798804+0000","last_change":"2024-03-29T04:07:41.426187+0000","last_active":"2024-03-29T04:07:42.798804+0000","last_peered":"2024-03-29T04:07:42.798804+0000","last_clean":"2024-03-29T04:07:42.798804+0000","last_became_active":"2024-03-29T04:07:41.425926+0000","last_became_peered":"2024-03-29T04:07:41.425926+0000","last_unstale":"2024-03-29T04:07:42.798804+0000","last_undegraded":"2024-03-29T04:07:42.798804+0000","last_fullsized":"2024-03-29T04:07:42.798804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:08.125575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797558+0000","last_change":"2024-03-29T04:07:40.430187+0000","last_active":"2024-03-29T04:07:42.797558+0000","last_peered":"2024-03-29T04:07:42.797558+0000","last_clean":"2024-03-29T04:07:42.797558+0000","last_became_active":"2024-03-29T04:07:40.429878+0000","last_became_peered":"2024-03-29T04:07:40.429878+0000","last_unstale":"2024-03-29T04:07:42.797558+0000","last_undegraded":"2024-03-29T04:07:42.797558+0000","last_fullsized":"2024-03-29T04:07:42.797558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413340+0000","last_change":"2024-03-29T04:07:41.427276+0000","last_active":"2024-03-29T04:07:42.413340+0000","last_peered":"2024-03-29T04:07:42.413340+0000","last_clean":"2024-03-29T04:07:42.413340+0000","last_became_active":"2024-03-29T04:07:41.427019+0000","last_became_peered":"2024-03-29T04:07:41.427019+0000","last_unstale":"2024-03-29T04:07:42.413340+0000","last_undegraded":"2024-03-29T04:07:42.413340+0000","last_fullsized":"2024-03-29T04:07:42.413340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:31:27.159107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.416916+0000","last_change":"2024-03-29T04:07:40.430476+0000","last_active":"2024-03-29T04:07:42.416916+0000","last_peered":"2024-03-29T04:07:42.416916+0000","last_clean":"2024-03-29T04:07:42.416916+0000","last_became_active":"2024-03-29T04:07:40.430169+0000","last_became_peered":"2024-03-29T04:07:40.430169+0000","last_unstale":"2024-03-29T04:07:42.416916+0000","last_undegraded":"2024-03-29T04:07:42.416916+0000","last_fullsized":"2024-03-29T04:07:42.416916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798544+0000","last_change":"2024-03-29T04:07:41.439222+0000","last_active":"2024-03-29T04:07:42.798544+0000","last_peered":"2024-03-29T04:07:42.798544+0000","last_clean":"2024-03-29T04:07:42.798544+0000","last_became_active":"2024-03-29T04:07:41.437292+0000","last_became_peered":"2024-03-29T04:07:41.437292+0000","last_unstale":"2024-03-29T04:07:42.798544+0000","last_undegraded":"2024-03-29T04:07:42.798544+0000","last_fullsized":"2024-03-29T04:07:42.798544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:13:07.317381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798565+0000","last_change":"2024-03-29T04:07:40.425014+0000","last_active":"2024-03-29T04:07:42.798565+0000","last_peered":"2024-03-29T04:07:42.798565+0000","last_clean":"2024-03-29T04:07:42.798565+0000","last_became_active":"2024-03-29T04:07:40.424254+0000","last_became_peered":"2024-03-29T04:07:40.424254+0000","last_unstale":"2024-03-29T04:07:42.798565+0000","last_undegraded":"2024-03-29T04:07:42.798565+0000","last_fullsized":"2024-03-29T04:07:42.798565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:08.909004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797721+0000","last_change":"2024-03-29T04:07:41.436336+0000","last_active":"2024-03-29T04:07:42.797721+0000","last_peered":"2024-03-29T04:07:42.797721+0000","last_clean":"2024-03-29T04:07:42.797721+0000","last_became_active":"2024-03-29T04:07:41.436089+0000","last_became_peered":"2024-03-29T04:07:41.436089+0000","last_unstale":"2024-03-29T04:07:42.797721+0000","last_undegraded":"2024-03-29T04:07:42.797721+0000","last_fullsized":"2024-03-29T04:07:42.797721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:19.861306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413545+0000","last_change":"2024-03-29T04:07:40.424800+0000","last_active":"2024-03-29T04:07:42.413545+0000","last_peered":"2024-03-29T04:07:42.413545+0000","last_clean":"2024-03-29T04:07:42.413545+0000","last_became_active":"2024-03-29T04:07:40.424508+0000","last_became_peered":"2024-03-29T04:07:40.424508+0000","last_unstale":"2024-03-29T04:07:42.413545+0000","last_undegraded":"2024-03-29T04:07:42.413545+0000","last_fullsized":"2024-03-29T04:07:42.413545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:44.678749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417383+0000","last_change":"2024-03-29T04:07:41.429324+0000","last_active":"2024-03-29T04:07:42.417383+0000","last_peered":"2024-03-29T04:07:42.417383+0000","last_clean":"2024-03-29T04:07:42.417383+0000","last_became_active":"2024-03-29T04:07:41.428117+0000","last_became_peered":"2024-03-29T04:07:41.428117+0000","last_unstale":"2024-03-29T04:07:42.417383+0000","last_undegraded":"2024-03-29T04:07:42.417383+0000","last_fullsized":"2024-03-29T04:07:42.417383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:45.683611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417360+0000","last_change":"2024-03-29T04:07:40.428764+0000","last_active":"2024-03-29T04:07:42.417360+0000","last_peered":"2024-03-29T04:07:42.417360+0000","last_clean":"2024-03-29T04:07:42.417360+0000","last_became_active":"2024-03-29T04:07:40.428510+0000","last_became_peered":"2024-03-29T04:07:40.428510+0000","last_unstale":"2024-03-29T04:07:42.417360+0000","last_undegraded":"2024-03-29T04:07:42.417360+0000","last_fullsized":"2024-03-29T04:07:42.417360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:34.960075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798009+0000","last_change":"2024-03-29T04:07:41.431774+0000","last_active":"2024-03-29T04:07:42.798009+0000","last_peered":"2024-03-29T04:07:42.798009+0000","last_clean":"2024-03-29T04:07:42.798009+0000","last_became_active":"2024-03-29T04:07:41.431503+0000","last_became_peered":"2024-03-29T04:07:41.431503+0000","last_unstale":"2024-03-29T04:07:42.798009+0000","last_undegraded":"2024-03-29T04:07:42.798009+0000","last_fullsized":"2024-03-29T04:07:42.798009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:53:03.772026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798322+0000","last_change":"2024-03-29T04:07:40.418935+0000","last_active":"2024-03-29T04:07:42.798322+0000","last_peered":"2024-03-29T04:07:42.798322+0000","last_clean":"2024-03-29T04:07:42.798322+0000","last_became_active":"2024-03-29T04:07:40.418678+0000","last_became_peered":"2024-03-29T04:07:40.418678+0000","last_unstale":"2024-03-29T04:07:42.798322+0000","last_undegraded":"2024-03-29T04:07:42.798322+0000","last_fullsized":"2024-03-29T04:07:42.798322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:06:18.089762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798208+0000","last_change":"2024-03-29T04:07:41.429488+0000","last_active":"2024-03-29T04:07:42.798208+0000","last_peered":"2024-03-29T04:07:42.798208+0000","last_clean":"2024-03-29T04:07:42.798208+0000","last_became_active":"2024-03-29T04:07:41.429233+0000","last_became_peered":"2024-03-29T04:07:41.429233+0000","last_unstale":"2024-03-29T04:07:42.798208+0000","last_undegraded":"2024-03-29T04:07:42.798208+0000","last_fullsized":"2024-03-29T04:07:42.798208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:28:53.046219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797793+0000","last_change":"2024-03-29T04:07:40.429229+0000","last_active":"2024-03-29T04:07:42.797793+0000","last_peered":"2024-03-29T04:07:42.797793+0000","last_clean":"2024-03-29T04:07:42.797793+0000","last_became_active":"2024-03-29T04:07:40.428949+0000","last_became_peered":"2024-03-29T04:07:40.428949+0000","last_unstale":"2024-03-29T04:07:42.797793+0000","last_undegraded":"2024-03-29T04:07:42.797793+0000","last_fullsized":"2024-03-29T04:07:42.797793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:11.556289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798285+0000","last_change":"2024-03-29T04:07:41.433525+0000","last_active":"2024-03-29T04:07:42.798285+0000","last_peered":"2024-03-29T04:07:42.798285+0000","last_clean":"2024-03-29T04:07:42.798285+0000","last_became_active":"2024-03-29T04:07:41.433128+0000","last_became_peered":"2024-03-29T04:07:41.433128+0000","last_unstale":"2024-03-29T04:07:42.798285+0000","last_undegraded":"2024-03-29T04:07:42.798285+0000","last_fullsized":"2024-03-29T04:07:42.798285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:14.560489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.416530+0000","last_change":"2024-03-29T04:07:40.419079+0000","last_active":"2024-03-29T04:07:42.416530+0000","last_peered":"2024-03-29T04:07:42.416530+0000","last_clean":"2024-03-29T04:07:42.416530+0000","last_became_active":"2024-03-29T04:07:40.418589+0000","last_became_peered":"2024-03-29T04:07:40.418589+0000","last_unstale":"2024-03-29T04:07:42.416530+0000","last_undegraded":"2024-03-29T04:07:42.416530+0000","last_fullsized":"2024-03-29T04:07:42.416530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:11.556289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417165+0000","last_change":"2024-03-29T04:07:41.425671+0000","last_active":"2024-03-29T04:07:42.417165+0000","last_peered":"2024-03-29T04:07:42.417165+0000","last_clean":"2024-03-29T04:07:42.417165+0000","last_became_active":"2024-03-29T04:07:41.425336+0000","last_became_peered":"2024-03-29T04:07:41.425336+0000","last_unstale":"2024-03-29T04:07:42.417165+0000","last_undegraded":"2024-03-29T04:07:42.417165+0000","last_fullsized":"2024-03-29T04:07:42.417165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:10:59.508554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797485+0000","last_change":"2024-03-29T04:07:40.419997+0000","last_active":"2024-03-29T04:07:42.797485+0000","last_peered":"2024-03-29T04:07:42.797485+0000","last_clean":"2024-03-29T04:07:42.797485+0000","last_became_active":"2024-03-29T04:07:40.419725+0000","last_became_peered":"2024-03-29T04:07:40.419725+0000","last_unstale":"2024-03-29T04:07:42.797485+0000","last_undegraded":"2024-03-29T04:07:42.797485+0000","last_fullsized":"2024-03-29T04:07:42.797485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:40:12.105363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797775+0000","last_change":"2024-03-29T04:07:41.429335+0000","last_active":"2024-03-29T04:07:42.797775+0000","last_peered":"2024-03-29T04:07:42.797775+0000","last_clean":"2024-03-29T04:07:42.797775+0000","last_became_active":"2024-03-29T04:07:41.428990+0000","last_became_peered":"2024-03-29T04:07:41.428990+0000","last_unstale":"2024-03-29T04:07:42.797775+0000","last_undegraded":"2024-03-29T04:07:42.797775+0000","last_fullsized":"2024-03-29T04:07:42.797775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:58.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.418057+0000","last_change":"2024-03-29T04:07:40.422355+0000","last_active":"2024-03-29T04:07:42.418057+0000","last_peered":"2024-03-29T04:07:42.418057+0000","last_clean":"2024-03-29T04:07:42.418057+0000","last_became_active":"2024-03-29T04:07:40.422081+0000","last_became_peered":"2024-03-29T04:07:40.422081+0000","last_unstale":"2024-03-29T04:07:42.418057+0000","last_undegraded":"2024-03-29T04:07:42.418057+0000","last_fullsized":"2024-03-29T04:07:42.418057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:42.944057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798226+0000","last_change":"2024-03-29T04:07:41.428540+0000","last_active":"2024-03-29T04:07:42.798226+0000","last_peered":"2024-03-29T04:07:42.798226+0000","last_clean":"2024-03-29T04:07:42.798226+0000","last_became_active":"2024-03-29T04:07:41.428266+0000","last_became_peered":"2024-03-29T04:07:41.428266+0000","last_unstale":"2024-03-29T04:07:42.798226+0000","last_undegraded":"2024-03-29T04:07:42.798226+0000","last_fullsized":"2024-03-29T04:07:42.798226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:13:07.317381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797957+0000","last_change":"2024-03-29T04:07:40.423940+0000","last_active":"2024-03-29T04:07:42.797957+0000","last_peered":"2024-03-29T04:07:42.797957+0000","last_clean":"2024-03-29T04:07:42.797957+0000","last_became_active":"2024-03-29T04:07:40.423675+0000","last_became_peered":"2024-03-29T04:07:40.423675+0000","last_unstale":"2024-03-29T04:07:42.797957+0000","last_undegraded":"2024-03-29T04:07:42.797957+0000","last_fullsized":"2024-03-29T04:07:42.797957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:08.909004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798698+0000","last_change":"2024-03-29T04:07:41.429372+0000","last_active":"2024-03-29T04:07:42.798698+0000","last_peered":"2024-03-29T04:07:42.798698+0000","last_clean":"2024-03-29T04:07:42.798698+0000","last_became_active":"2024-03-29T04:07:41.429084+0000","last_became_peered":"2024-03-29T04:07:41.429084+0000","last_unstale":"2024-03-29T04:07:42.798698+0000","last_undegraded":"2024-03-29T04:07:42.798698+0000","last_fullsized":"2024-03-29T04:07:42.798698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:44:51.240215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413552+0000","last_change":"2024-03-29T04:07:40.420126+0000","last_active":"2024-03-29T04:07:42.413552+0000","last_peered":"2024-03-29T04:07:42.413552+0000","last_clean":"2024-03-29T04:07:42.413552+0000","last_became_active":"2024-03-29T04:07:40.419852+0000","last_became_peered":"2024-03-29T04:07:40.419852+0000","last_unstale":"2024-03-29T04:07:42.413552+0000","last_undegraded":"2024-03-29T04:07:42.413552+0000","last_fullsized":"2024-03-29T04:07:42.413552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:41.833725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798305+0000","last_change":"2024-03-29T04:07:41.435921+0000","last_active":"2024-03-29T04:07:42.798305+0000","last_peered":"2024-03-29T04:07:42.798305+0000","last_clean":"2024-03-29T04:07:42.798305+0000","last_became_active":"2024-03-29T04:07:41.435681+0000","last_became_peered":"2024-03-29T04:07:41.435681+0000","last_unstale":"2024-03-29T04:07:42.798305+0000","last_undegraded":"2024-03-29T04:07:42.798305+0000","last_fullsized":"2024-03-29T04:07:42.798305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:28:45.871462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797831+0000","last_change":"2024-03-29T04:07:40.430102+0000","last_active":"2024-03-29T04:07:42.797831+0000","last_peered":"2024-03-29T04:07:42.797831+0000","last_clean":"2024-03-29T04:07:42.797831+0000","last_became_active":"2024-03-29T04:07:40.429846+0000","last_became_peered":"2024-03-29T04:07:40.429846+0000","last_unstale":"2024-03-29T04:07:42.797831+0000","last_undegraded":"2024-03-29T04:07:42.797831+0000","last_fullsized":"2024-03-29T04:07:42.797831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798212+0000","last_change":"2024-03-29T04:07:41.438935+0000","last_active":"2024-03-29T04:07:42.798212+0000","last_peered":"2024-03-29T04:07:42.798212+0000","last_clean":"2024-03-29T04:07:42.798212+0000","last_became_active":"2024-03-29T04:07:41.438629+0000","last_became_peered":"2024-03-29T04:07:41.438629+0000","last_unstale":"2024-03-29T04:07:42.798212+0000","last_undegraded":"2024-03-29T04:07:42.798212+0000","last_fullsized":"2024-03-29T04:07:42.798212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:28:45.871462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797615+0000","last_change":"2024-03-29T04:07:29.856799+0000","last_active":"2024-03-29T04:07:42.797615+0000","last_peered":"2024-03-29T04:07:42.797615+0000","last_clean":"2024-03-29T04:07:42.797615+0000","last_became_active":"2024-03-29T04:07:29.855873+0000","last_became_peered":"2024-03-29T04:07:29.855873+0000","last_unstale":"2024-03-29T04:07:42.797615+0000","last_undegraded":"2024-03-29T04:07:42.797615+0000","last_fullsized":"2024-03-29T04:07:42.797615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_clean_scrub_stamp":"2024-03-29T04:07:28.661516+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:25:15.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797863+0000","last_change":"2024-03-29T04:07:40.427444+0000","last_active":"2024-03-29T04:07:42.797863+0000","last_peered":"2024-03-29T04:07:42.797863+0000","last_clean":"2024-03-29T04:07:42.797863+0000","last_became_active":"2024-03-29T04:07:40.426408+0000","last_became_peered":"2024-03-29T04:07:40.426408+0000","last_unstale":"2024-03-29T04:07:42.797863+0000","last_undegraded":"2024-03-29T04:07:42.797863+0000","last_fullsized":"2024-03-29T04:07:42.797863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:04.924835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798093+0000","last_change":"2024-03-29T04:07:41.441256+0000","last_active":"2024-03-29T04:07:42.798093+0000","last_peered":"2024-03-29T04:07:42.798093+0000","last_clean":"2024-03-29T04:07:42.798093+0000","last_became_active":"2024-03-29T04:07:41.441002+0000","last_became_peered":"2024-03-29T04:07:41.441002+0000","last_unstale":"2024-03-29T04:07:42.798093+0000","last_undegraded":"2024-03-29T04:07:42.798093+0000","last_fullsized":"2024-03-29T04:07:42.798093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:57.438333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797519+0000","last_change":"2024-03-29T04:07:40.427239+0000","last_active":"2024-03-29T04:07:42.797519+0000","last_peered":"2024-03-29T04:07:42.797519+0000","last_clean":"2024-03-29T04:07:42.797519+0000","last_became_active":"2024-03-29T04:07:40.426943+0000","last_became_peered":"2024-03-29T04:07:40.426943+0000","last_unstale":"2024-03-29T04:07:42.797519+0000","last_undegraded":"2024-03-29T04:07:42.797519+0000","last_fullsized":"2024-03-29T04:07:42.797519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:50:50.615524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797844+0000","last_change":"2024-03-29T04:07:41.433014+0000","last_active":"2024-03-29T04:07:42.797844+0000","last_peered":"2024-03-29T04:07:42.797844+0000","last_clean":"2024-03-29T04:07:42.797844+0000","last_became_active":"2024-03-29T04:07:41.432678+0000","last_became_peered":"2024-03-29T04:07:41.432678+0000","last_unstale":"2024-03-29T04:07:42.797844+0000","last_undegraded":"2024-03-29T04:07:42.797844+0000","last_fullsized":"2024-03-29T04:07:42.797844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:10:59.508554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797581+0000","last_change":"2024-03-29T04:07:40.421624+0000","last_active":"2024-03-29T04:07:42.797581+0000","last_peered":"2024-03-29T04:07:42.797581+0000","last_clean":"2024-03-29T04:07:42.797581+0000","last_became_active":"2024-03-29T04:07:40.421376+0000","last_became_peered":"2024-03-29T04:07:40.421376+0000","last_unstale":"2024-03-29T04:07:42.797581+0000","last_undegraded":"2024-03-29T04:07:42.797581+0000","last_fullsized":"2024-03-29T04:07:42.797581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:56:04.924835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413533+0000","last_change":"2024-03-29T04:07:41.421588+0000","last_active":"2024-03-29T04:07:42.413533+0000","last_peered":"2024-03-29T04:07:42.413533+0000","last_clean":"2024-03-29T04:07:42.413533+0000","last_became_active":"2024-03-29T04:07:41.421329+0000","last_became_peered":"2024-03-29T04:07:41.421329+0000","last_unstale":"2024-03-29T04:07:42.413533+0000","last_undegraded":"2024-03-29T04:07:42.413533+0000","last_fullsized":"2024-03-29T04:07:42.413533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:35.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797840+0000","last_change":"2024-03-29T04:07:40.424596+0000","last_active":"2024-03-29T04:07:42.797840+0000","last_peered":"2024-03-29T04:07:42.797840+0000","last_clean":"2024-03-29T04:07:42.797840+0000","last_became_active":"2024-03-29T04:07:40.424288+0000","last_became_peered":"2024-03-29T04:07:40.424288+0000","last_unstale":"2024-03-29T04:07:42.797840+0000","last_undegraded":"2024-03-29T04:07:42.797840+0000","last_fullsized":"2024-03-29T04:07:42.797840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:34.960075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.418058+0000","last_change":"2024-03-29T04:07:41.429654+0000","last_active":"2024-03-29T04:07:42.418058+0000","last_peered":"2024-03-29T04:07:42.418058+0000","last_clean":"2024-03-29T04:07:42.418058+0000","last_became_active":"2024-03-29T04:07:41.429335+0000","last_became_peered":"2024-03-29T04:07:41.429335+0000","last_unstale":"2024-03-29T04:07:42.418058+0000","last_undegraded":"2024-03-29T04:07:42.418058+0000","last_fullsized":"2024-03-29T04:07:42.418058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:50:51.620387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413527+0000","last_change":"2024-03-29T04:07:40.420207+0000","last_active":"2024-03-29T04:07:42.413527+0000","last_peered":"2024-03-29T04:07:42.413527+0000","last_clean":"2024-03-29T04:07:42.413527+0000","last_became_active":"2024-03-29T04:07:40.419936+0000","last_became_peered":"2024-03-29T04:07:40.419936+0000","last_unstale":"2024-03-29T04:07:42.413527+0000","last_undegraded":"2024-03-29T04:07:42.413527+0000","last_fullsized":"2024-03-29T04:07:42.413527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:04:11.556289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798302+0000","last_change":"2024-03-29T04:07:41.433871+0000","last_active":"2024-03-29T04:07:42.798302+0000","last_peered":"2024-03-29T04:07:42.798302+0000","last_clean":"2024-03-29T04:07:42.798302+0000","last_became_active":"2024-03-29T04:07:41.433183+0000","last_became_peered":"2024-03-29T04:07:41.433183+0000","last_unstale":"2024-03-29T04:07:42.798302+0000","last_undegraded":"2024-03-29T04:07:42.798302+0000","last_fullsized":"2024-03-29T04:07:42.798302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:27:35.869429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798386+0000","last_change":"2024-03-29T04:07:40.432156+0000","last_active":"2024-03-29T04:07:42.798386+0000","last_peered":"2024-03-29T04:07:42.798386+0000","last_clean":"2024-03-29T04:07:42.798386+0000","last_became_active":"2024-03-29T04:07:40.431840+0000","last_became_peered":"2024-03-29T04:07:40.431840+0000","last_unstale":"2024-03-29T04:07:42.798386+0000","last_undegraded":"2024-03-29T04:07:42.798386+0000","last_fullsized":"2024-03-29T04:07:42.798386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:36:02.627626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413428+0000","last_change":"2024-03-29T04:07:41.421216+0000","last_active":"2024-03-29T04:07:42.413428+0000","last_peered":"2024-03-29T04:07:42.413428+0000","last_clean":"2024-03-29T04:07:42.413428+0000","last_became_active":"2024-03-29T04:07:41.420934+0000","last_became_peered":"2024-03-29T04:07:41.420934+0000","last_unstale":"2024-03-29T04:07:42.413428+0000","last_undegraded":"2024-03-29T04:07:42.413428+0000","last_fullsized":"2024-03-29T04:07:42.413428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:57.438333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.433094+0000","last_change":"2024-03-29T04:07:40.421842+0000","last_active":"2024-03-29T04:07:42.433094+0000","last_peered":"2024-03-29T04:07:42.433094+0000","last_clean":"2024-03-29T04:07:42.433094+0000","last_became_active":"2024-03-29T04:07:40.421590+0000","last_became_peered":"2024-03-29T04:07:40.421590+0000","last_unstale":"2024-03-29T04:07:42.433094+0000","last_undegraded":"2024-03-29T04:07:42.433094+0000","last_fullsized":"2024-03-29T04:07:42.433094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:08.909004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798399+0000","last_change":"2024-03-29T04:07:41.429458+0000","last_active":"2024-03-29T04:07:42.798399+0000","last_peered":"2024-03-29T04:07:42.798399+0000","last_clean":"2024-03-29T04:07:42.798399+0000","last_became_active":"2024-03-29T04:07:41.429115+0000","last_became_peered":"2024-03-29T04:07:41.429115+0000","last_unstale":"2024-03-29T04:07:42.798399+0000","last_undegraded":"2024-03-29T04:07:42.798399+0000","last_fullsized":"2024-03-29T04:07:42.798399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:35.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417036+0000","last_change":"2024-03-29T04:07:40.422954+0000","last_active":"2024-03-29T04:07:42.417036+0000","last_peered":"2024-03-29T04:07:42.417036+0000","last_clean":"2024-03-29T04:07:42.417036+0000","last_became_active":"2024-03-29T04:07:40.422548+0000","last_became_peered":"2024-03-29T04:07:40.422548+0000","last_unstale":"2024-03-29T04:07:42.417036+0000","last_undegraded":"2024-03-29T04:07:42.417036+0000","last_fullsized":"2024-03-29T04:07:42.417036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:34:08.909004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798579+0000","last_change":"2024-03-29T04:07:41.425089+0000","last_active":"2024-03-29T04:07:42.798579+0000","last_peered":"2024-03-29T04:07:42.798579+0000","last_clean":"2024-03-29T04:07:42.798579+0000","last_became_active":"2024-03-29T04:07:41.424814+0000","last_became_peered":"2024-03-29T04:07:41.424814+0000","last_unstale":"2024-03-29T04:07:42.798579+0000","last_undegraded":"2024-03-29T04:07:42.798579+0000","last_fullsized":"2024-03-29T04:07:42.798579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:40.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797416+0000","last_change":"2024-03-29T04:07:40.429808+0000","last_active":"2024-03-29T04:07:42.797416+0000","last_peered":"2024-03-29T04:07:42.797416+0000","last_clean":"2024-03-29T04:07:42.797416+0000","last_became_active":"2024-03-29T04:07:40.428865+0000","last_became_peered":"2024-03-29T04:07:40.428865+0000","last_unstale":"2024-03-29T04:07:42.797416+0000","last_undegraded":"2024-03-29T04:07:42.797416+0000","last_fullsized":"2024-03-29T04:07:42.797416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:13:44.678749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417729+0000","last_change":"2024-03-29T04:07:41.434698+0000","last_active":"2024-03-29T04:07:42.417729+0000","last_peered":"2024-03-29T04:07:42.417729+0000","last_clean":"2024-03-29T04:07:42.417729+0000","last_became_active":"2024-03-29T04:07:41.434270+0000","last_became_peered":"2024-03-29T04:07:41.434270+0000","last_unstale":"2024-03-29T04:07:42.417729+0000","last_undegraded":"2024-03-29T04:07:42.417729+0000","last_fullsized":"2024-03-29T04:07:42.417729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:42.838588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413481+0000","last_change":"2024-03-29T04:07:40.425280+0000","last_active":"2024-03-29T04:07:42.413481+0000","last_peered":"2024-03-29T04:07:42.413481+0000","last_clean":"2024-03-29T04:07:42.413481+0000","last_became_active":"2024-03-29T04:07:40.424998+0000","last_became_peered":"2024-03-29T04:07:40.424998+0000","last_unstale":"2024-03-29T04:07:42.413481+0000","last_undegraded":"2024-03-29T04:07:42.413481+0000","last_fullsized":"2024-03-29T04:07:42.413481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:35:34.960075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798861+0000","last_change":"2024-03-29T04:07:41.424267+0000","last_active":"2024-03-29T04:07:42.798861+0000","last_peered":"2024-03-29T04:07:42.798861+0000","last_clean":"2024-03-29T04:07:42.798861+0000","last_became_active":"2024-03-29T04:07:41.423920+0000","last_became_peered":"2024-03-29T04:07:41.423920+0000","last_unstale":"2024-03-29T04:07:42.798861+0000","last_undegraded":"2024-03-29T04:07:42.798861+0000","last_fullsized":"2024-03-29T04:07:42.798861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:32:16.919913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.413416+0000","last_change":"2024-03-29T04:07:40.422602+0000","last_active":"2024-03-29T04:07:42.413416+0000","last_peered":"2024-03-29T04:07:42.413416+0000","last_clean":"2024-03-29T04:07:42.413416+0000","last_became_active":"2024-03-29T04:07:40.422332+0000","last_became_peered":"2024-03-29T04:07:40.422332+0000","last_unstale":"2024-03-29T04:07:42.413416+0000","last_undegraded":"2024-03-29T04:07:42.413416+0000","last_fullsized":"2024-03-29T04:07:42.413416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.799042+0000","last_change":"2024-03-29T04:07:41.427166+0000","last_active":"2024-03-29T04:07:42.799042+0000","last_peered":"2024-03-29T04:07:42.799042+0000","last_clean":"2024-03-29T04:07:42.799042+0000","last_became_active":"2024-03-29T04:07:41.426914+0000","last_became_peered":"2024-03-29T04:07:41.426914+0000","last_unstale":"2024-03-29T04:07:42.799042+0000","last_undegraded":"2024-03-29T04:07:42.799042+0000","last_fullsized":"2024-03-29T04:07:42.799042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:58.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798369+0000","last_change":"2024-03-29T04:07:40.430896+0000","last_active":"2024-03-29T04:07:42.798369+0000","last_peered":"2024-03-29T04:07:42.798369+0000","last_clean":"2024-03-29T04:07:42.798369+0000","last_became_active":"2024-03-29T04:07:40.430549+0000","last_became_peered":"2024-03-29T04:07:40.430549+0000","last_unstale":"2024-03-29T04:07:42.798369+0000","last_undegraded":"2024-03-29T04:07:42.798369+0000","last_fullsized":"2024-03-29T04:07:42.798369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:39:46.902638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.417189+0000","last_change":"2024-03-29T04:07:41.429537+0000","last_active":"2024-03-29T04:07:42.417189+0000","last_peered":"2024-03-29T04:07:42.417189+0000","last_clean":"2024-03-29T04:07:42.417189+0000","last_became_active":"2024-03-29T04:07:41.429173+0000","last_became_peered":"2024-03-29T04:07:41.429173+0000","last_unstale":"2024-03-29T04:07:42.417189+0000","last_undegraded":"2024-03-29T04:07:42.417189+0000","last_fullsized":"2024-03-29T04:07:42.417189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:44:57.438333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798597+0000","last_change":"2024-03-29T04:07:41.436387+0000","last_active":"2024-03-29T04:07:42.798597+0000","last_peered":"2024-03-29T04:07:42.798597+0000","last_clean":"2024-03-29T04:07:42.798597+0000","last_became_active":"2024-03-29T04:07:41.436043+0000","last_became_peered":"2024-03-29T04:07:41.436043+0000","last_unstale":"2024-03-29T04:07:42.798597+0000","last_undegraded":"2024-03-29T04:07:42.798597+0000","last_fullsized":"2024-03-29T04:07:42.798597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:12:50.176893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798007+0000","last_change":"2024-03-29T04:07:40.431266+0000","last_active":"2024-03-29T04:07:42.798007+0000","last_peered":"2024-03-29T04:07:42.798007+0000","last_clean":"2024-03-29T04:07:42.798007+0000","last_became_active":"2024-03-29T04:07:40.430984+0000","last_became_peered":"2024-03-29T04:07:40.430984+0000","last_unstale":"2024-03-29T04:07:42.798007+0000","last_undegraded":"2024-03-29T04:07:42.798007+0000","last_fullsized":"2024-03-29T04:07:42.798007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:41.833725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798023+0000","last_change":"2024-03-29T04:07:41.436711+0000","last_active":"2024-03-29T04:07:42.798023+0000","last_peered":"2024-03-29T04:07:42.798023+0000","last_clean":"2024-03-29T04:07:42.798023+0000","last_became_active":"2024-03-29T04:07:41.436145+0000","last_became_peered":"2024-03-29T04:07:41.436145+0000","last_unstale":"2024-03-29T04:07:42.798023+0000","last_undegraded":"2024-03-29T04:07:42.798023+0000","last_fullsized":"2024-03-29T04:07:42.798023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:58.588169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798296+0000","last_change":"2024-03-29T04:07:40.430774+0000","last_active":"2024-03-29T04:07:42.798296+0000","last_peered":"2024-03-29T04:07:42.798296+0000","last_clean":"2024-03-29T04:07:42.798296+0000","last_became_active":"2024-03-29T04:07:40.430503+0000","last_became_peered":"2024-03-29T04:07:40.430503+0000","last_unstale":"2024-03-29T04:07:42.798296+0000","last_undegraded":"2024-03-29T04:07:42.798296+0000","last_fullsized":"2024-03-29T04:07:42.798296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:37.084723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797696+0000","last_change":"2024-03-29T04:07:41.441323+0000","last_active":"2024-03-29T04:07:42.797696+0000","last_peered":"2024-03-29T04:07:42.797696+0000","last_clean":"2024-03-29T04:07:42.797696+0000","last_became_active":"2024-03-29T04:07:41.441074+0000","last_became_peered":"2024-03-29T04:07:41.441074+0000","last_unstale":"2024-03-29T04:07:42.797696+0000","last_undegraded":"2024-03-29T04:07:42.797696+0000","last_fullsized":"2024-03-29T04:07:42.797696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:36:03.632489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798948+0000","last_change":"2024-03-29T04:07:40.431979+0000","last_active":"2024-03-29T04:07:42.798948+0000","last_peered":"2024-03-29T04:07:42.798948+0000","last_clean":"2024-03-29T04:07:42.798948+0000","last_became_active":"2024-03-29T04:07:40.431702+0000","last_became_peered":"2024-03-29T04:07:40.431702+0000","last_unstale":"2024-03-29T04:07:42.798948+0000","last_undegraded":"2024-03-29T04:07:42.798948+0000","last_fullsized":"2024-03-29T04:07:42.798948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:40.462024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.798867+0000","last_change":"2024-03-29T04:07:41.427017+0000","last_active":"2024-03-29T04:07:42.798867+0000","last_peered":"2024-03-29T04:07:42.798867+0000","last_clean":"2024-03-29T04:07:42.798867+0000","last_became_active":"2024-03-29T04:07:41.426749+0000","last_became_peered":"2024-03-29T04:07:41.426749+0000","last_unstale":"2024-03-29T04:07:42.798867+0000","last_undegraded":"2024-03-29T04:07:42.798867+0000","last_fullsized":"2024-03-29T04:07:42.798867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:40.403565+0000","last_clean_scrub_stamp":"2024-03-29T04:07:40.403565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:27:37.837537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-29T04:07:42.797804+0000","last_change":"2024-03-29T04:07:40.421536+0000","last_active":"2024-03-29T04:07:42.797804+0000","last_peered":"2024-03-29T04:07:42.797804+0000","last_clean":"2024-03-29T04:07:42.797804+0000","last_became_active":"2024-03-29T04:07:40.421279+0000","last_became_peered":"2024-03-29T04:07:40.421279+0000","last_unstale":"2024-03-29T04:07:42.797804+0000","last_undegraded":"2024-03-29T04:07:42.797804+0000","last_fullsized":"2024-03-29T04:07:42.797804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:39.398702+0000","last_clean_scrub_stamp":"2024-03-29T04:07:39.398702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:43.704154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:07:49.062 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-29T04:07:49.062 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-29T04:07:49.062 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-29T04:07:49.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-29T04:07:49.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.218+0000 7f469a900640 1 Processor -- start 2024-03-29T04:07:49.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.218+0000 7f469a900640 1 -- start start 2024-03-29T04:07:49.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f469a900640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f469415e880 0x7f46940cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f469a900640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46940b4340 0x7f46940d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f469a900640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46940cf7f0 con 0x7f469415ec60 2024-03-29T04:07:49.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f469a900640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f46940cfe00 con 0x7f469415e880 2024-03-29T04:07:49.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f469a900640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f46940cea70 con 0x7f46940b4340 2024-03-29T04:07:49.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f4693fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f469415e880 0x7f46940cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f46937fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f469415ec60 0x7f46940ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:40500/0 (socket says 172.21.15.97:40500) 2024-03-29T04:07:49.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f4693fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f469415e880 0x7f46940cdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:55768/0 (socket says 172.21.15.97:55768) 2024-03-29T04:07:49.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f46937fe640 1 -- 172.21.15.97:0/2483795753 learned_addr learned my addr 172.21.15.97:0/2483795753 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:49.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f4698e9d640 1 --2- 172.21.15.97:0/2483795753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46940b4340 0x7f46940d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f4693fff640 1 -- 172.21.15.97:0/2483795753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46940b4340 msgr2=0x7f46940d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:49.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f4693fff640 1 --2- 172.21.15.97:0/2483795753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46940b4340 0x7f46940d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f4693fff640 1 -- 172.21.15.97:0/2483795753 >> v1:172.21.15.154:6789/0 conn(0x7f469415ec60 legacy=0x7f46940ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f4693fff640 1 -- 172.21.15.97:0/2483795753 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46940ced50 con 0x7f469415e880 2024-03-29T04:07:49.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f4692ffd640 1 -- 172.21.15.97:0/2483795753 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3407586755 0 0) 0x7f46940cf7f0 con 0x7f469415ec60 2024-03-29T04:07:49.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.219+0000 7f4698e9d640 1 --2- 172.21.15.97:0/2483795753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46940b4340 0x7f46940d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.220+0000 7f4693fff640 1 --2- 172.21.15.97:0/2483795753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f469415e880 0x7f46940cdd40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f46840097a0 tx=0x7f468402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd071feb69681f5f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.220+0000 7f4692ffd640 1 -- 172.21.15.97:0/2483795753 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684035020 con 0x7f469415e880 2024-03-29T04:07:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.220+0000 7f4692ffd640 1 -- 172.21.15.97:0/2483795753 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4684031830 con 0x7f469415e880 2024-03-29T04:07:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.220+0000 7f4692ffd640 1 -- 172.21.15.97:0/2483795753 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684031c00 con 0x7f469415e880 2024-03-29T04:07:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.221+0000 7f469a900640 1 -- 172.21.15.97:0/2483795753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f469415e880 msgr2=0x7f46940cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.221+0000 7f469a900640 1 --2- 172.21.15.97:0/2483795753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f469415e880 0x7f46940cdd40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f46840097a0 tx=0x7f468402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.221+0000 7f469a900640 1 -- 172.21.15.97:0/2483795753 shutdown_connections 2024-03-29T04:07:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.221+0000 7f469a900640 1 --2- 172.21.15.97:0/2483795753 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46940b4340 0x7f46940d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.221+0000 7f469a900640 1 --2- 172.21.15.97:0/2483795753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f469415e880 0x7f46940cdd40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.221+0000 7f469a900640 1 -- 172.21.15.97:0/2483795753 >> 172.21.15.97:0/2483795753 conn(0x7f46940b6860 msgr2=0x7f46940b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:49.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.221+0000 7f469a900640 1 -- 172.21.15.97:0/2483795753 shutdown_connections 2024-03-29T04:07:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.221+0000 7f469a900640 1 -- 172.21.15.97:0/2483795753 wait complete. 2024-03-29T04:07:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.222+0000 7f469a900640 1 Processor -- start 2024-03-29T04:07:49.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.222+0000 7f469a900640 1 -- start start 2024-03-29T04:07:49.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f469a900640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46940b4340 0x7f469415fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f469a900640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f469415e880 0x7f4694161960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f469a900640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f469415ec60 0x7f46941f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f469a900640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f46940d0950 con 0x7f46940b4340 2024-03-29T04:07:49.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f469a900640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f46940d07d0 con 0x7f469415ec60 2024-03-29T04:07:49.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f469a900640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f46940d0ad0 con 0x7f469415e880 2024-03-29T04:07:49.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f4693fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46940b4340 0x7f469415fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f4693fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46940b4340 0x7f469415fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39152/0 (socket says 172.21.15.97:39152) 2024-03-29T04:07:49.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f4693fff640 1 -- 172.21.15.97:0/1409107792 learned_addr learned my addr 172.21.15.97:0/1409107792 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:49.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f4698e9d640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f469415ec60 0x7f46941f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f46937fe640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f469415e880 0x7f4694161960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f4693fff640 1 -- 172.21.15.97:0/1409107792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f469415e880 msgr2=0x7f4694161960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f4693fff640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f469415e880 0x7f4694161960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f4693fff640 1 -- 172.21.15.97:0/1409107792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f469415ec60 msgr2=0x7f46941f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f4693fff640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f469415ec60 0x7f46941f3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f4693fff640 1 -- 172.21.15.97:0/1409107792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469415d020 con 0x7f46940b4340 2024-03-29T04:07:49.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f4693fff640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46940b4340 0x7f469415fa50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4684000c00 tx=0x7f46840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:49.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f4690ff9640 1 -- 172.21.15.97:0/1409107792 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684035070 con 0x7f46940b4340 2024-03-29T04:07:49.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f4690ff9640 1 -- 172.21.15.97:0/1409107792 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4684004230 con 0x7f46940b4340 2024-03-29T04:07:49.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f469a900640 1 -- 172.21.15.97:0/1409107792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469415d300 con 0x7f46940b4340 2024-03-29T04:07:49.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.223+0000 7f4690ff9640 1 -- 172.21.15.97:0/1409107792 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468403a630 con 0x7f46940b4340 2024-03-29T04:07:49.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.224+0000 7f469a900640 1 -- 172.21.15.97:0/1409107792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46941600e0 con 0x7f46940b4340 2024-03-29T04:07:49.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.224+0000 7f469a900640 1 -- 172.21.15.97:0/1409107792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4694068640 con 0x7f46940b4340 2024-03-29T04:07:49.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.225+0000 7f4690ff9640 1 -- 172.21.15.97:0/1409107792 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4684038020 con 0x7f46940b4340 2024-03-29T04:07:49.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.225+0000 7f4690ff9640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f468006c560 0x7f468006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.225+0000 7f4690ff9640 1 -- 172.21.15.97:0/1409107792 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f46840ad180 con 0x7f46940b4340 2024-03-29T04:07:49.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.225+0000 7f46937fe640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f468006c560 0x7f468006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.226+0000 7f46937fe640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f468006c560 0x7f468006ea20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f469415ab40 tx=0x7f4688007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:49.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.229+0000 7f4690ff9640 1 -- 172.21.15.97:0/1409107792 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4684040070 con 0x7f46940b4340 2024-03-29T04:07:49.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.442+0000 7f469a900640 1 -- 172.21.15.97:0/1409107792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4694070570 con 0x7f46940b4340 2024-03-29T04:07:49.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.443+0000 7f4690ff9640 1 -- 172.21.15.97:0/1409107792 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f468407b150 con 0x7f46940b4340 2024-03-29T04:07:49.423 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:49.423 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-29T04:07:49.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.445+0000 7f469a900640 1 -- 172.21.15.97:0/1409107792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f468006c560 msgr2=0x7f468006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.445+0000 7f469a900640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f468006c560 0x7f468006ea20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f469415ab40 tx=0x7f4688007de0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.445+0000 7f469a900640 1 -- 172.21.15.97:0/1409107792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46940b4340 msgr2=0x7f469415fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.445+0000 7f469a900640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46940b4340 0x7f469415fa50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4684000c00 tx=0x7f46840046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.445+0000 7f469a900640 1 -- 172.21.15.97:0/1409107792 shutdown_connections 2024-03-29T04:07:49.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.445+0000 7f469a900640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f468006c560 0x7f468006ea20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.445+0000 7f469a900640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f469415ec60 0x7f46941f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.445+0000 7f469a900640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f469415e880 0x7f4694161960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.445+0000 7f469a900640 1 --2- 172.21.15.97:0/1409107792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46940b4340 0x7f469415fa50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.445+0000 7f469a900640 1 -- 172.21.15.97:0/1409107792 >> 172.21.15.97:0/1409107792 conn(0x7f46940b6860 msgr2=0x7f469406f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:49.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.445+0000 7f469a900640 1 -- 172.21.15.97:0/1409107792 shutdown_connections 2024-03-29T04:07:49.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.445+0000 7f469a900640 1 -- 172.21.15.97:0/1409107792 wait complete. 2024-03-29T04:07:49.436 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-29T04:07:49.436 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.557+0000 7f7f1c463640 1 Processor -- start 2024-03-29T04:07:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.557+0000 7f7f1c463640 1 -- start start 2024-03-29T04:07:49.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.558+0000 7f7f1c463640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f140d0a00 0x7f7f140d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.558+0000 7f7f1c463640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f1415db70 con 0x7f7f140cfff0 2024-03-29T04:07:49.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.558+0000 7f7f1c463640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f14160fb0 con 0x7f7f1409ba40 2024-03-29T04:07:49.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.558+0000 7f7f1c463640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7f140cdf90 con 0x7f7f140d0a00 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.558+0000 7f7f199fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7f140cfff0 0x7f7f14160190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50488/0 (socket says 172.21.15.97:50488) 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.558+0000 7f7f199fe640 1 -- 172.21.15.97:0/2127853682 learned_addr learned my addr 172.21.15.97:0/2127853682 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.558+0000 7f7f1a1ff640 1 --2- 172.21.15.97:0/2127853682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f140d0a00 0x7f7f140d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.559+0000 7f7f191fd640 1 -- 172.21.15.97:0/2127853682 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1093813002 0 0) 0x7f7f1415db70 con 0x7f7f140cfff0 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.559+0000 7f7f191fd640 1 -- 172.21.15.97:0/2127853682 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f00003520 con 0x7f7f140cfff0 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.559+0000 7f7f191fd640 1 -- 172.21.15.97:0/2127853682 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136072914 0 0) 0x7f7f14160fb0 con 0x7f7f1409ba40 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.559+0000 7f7f191fd640 1 -- 172.21.15.97:0/2127853682 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f1415db70 con 0x7f7f1409ba40 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.559+0000 7f7f1a1ff640 1 -- 172.21.15.97:0/2127853682 >> v1:172.21.15.154:6789/0 conn(0x7f7f1409ba40 legacy=0x7f7f141608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.559+0000 7f7f1a1ff640 1 -- 172.21.15.97:0/2127853682 >> v1:172.21.15.97:6789/0 conn(0x7f7f140cfff0 legacy=0x7f7f14160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.559+0000 7f7f1a1ff640 1 -- 172.21.15.97:0/2127853682 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f140d1320 con 0x7f7f140d0a00 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.560+0000 7f7f1a1ff640 1 --2- 172.21.15.97:0/2127853682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f140d0a00 0x7f7f140d0de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f100097d0 tx=0x7f7f1002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5d197b32f392a05 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.560+0000 7f7f191fd640 1 -- 172.21.15.97:0/2127853682 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10035020 con 0x7f7f140d0a00 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.560+0000 7f7f191fd640 1 -- 172.21.15.97:0/2127853682 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f10031890 con 0x7f7f140d0a00 2024-03-29T04:07:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.560+0000 7f7f191fd640 1 -- 172.21.15.97:0/2127853682 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10031c40 con 0x7f7f140d0a00 2024-03-29T04:07:49.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.560+0000 7f7f1c463640 1 -- 172.21.15.97:0/2127853682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f140d0a00 msgr2=0x7f7f140d0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.560+0000 7f7f1c463640 1 --2- 172.21.15.97:0/2127853682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f140d0a00 0x7f7f140d0de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f100097d0 tx=0x7f7f1002b930 comp rx=0 tx=0).stop 2024-03-29T04:07:49.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.560+0000 7f7f1c463640 1 -- 172.21.15.97:0/2127853682 shutdown_connections 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.560+0000 7f7f1c463640 1 --2- 172.21.15.97:0/2127853682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f140d0a00 0x7f7f140d0de0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.560+0000 7f7f1c463640 1 -- 172.21.15.97:0/2127853682 >> 172.21.15.97:0/2127853682 conn(0x7f7f140b3f10 msgr2=0x7f7f140b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.560+0000 7f7f1c463640 1 -- 172.21.15.97:0/2127853682 shutdown_connections 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.560+0000 7f7f1c463640 1 -- 172.21.15.97:0/2127853682 wait complete. 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.562+0000 7f7f1c463640 1 Processor -- start 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.562+0000 7f7f1c463640 1 -- start start 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.562+0000 7f7f1c463640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f1409ba40 0x7f7f1409cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.562+0000 7f7f1c463640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f140cfff0 0x7f7f140cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.562+0000 7f7f1c463640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f140d0a00 0x7f7f140ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.562+0000 7f7f1c463640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f140cd940 con 0x7f7f1409ba40 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.562+0000 7f7f1c463640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f140cd660 con 0x7f7f140d0a00 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.562+0000 7f7f1c463640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7f140cdc20 con 0x7f7f140cfff0 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.562+0000 7f7f1a1ff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f1409ba40 0x7f7f1409cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.562+0000 7f7f1a1ff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f1409ba40 0x7f7f1409cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39164/0 (socket says 172.21.15.97:39164) 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.562+0000 7f7f1a1ff640 1 -- 172.21.15.97:0/1368844609 learned_addr learned my addr 172.21.15.97:0/1368844609 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.562+0000 7f7f199fe640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f140cfff0 0x7f7f140cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.562+0000 7f7f1aa00640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f140d0a00 0x7f7f140ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.563+0000 7f7f1a1ff640 1 -- 172.21.15.97:0/1368844609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f140cfff0 msgr2=0x7f7f140cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.563+0000 7f7f1a1ff640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f140cfff0 0x7f7f140cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.563+0000 7f7f1a1ff640 1 -- 172.21.15.97:0/1368844609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f140d0a00 msgr2=0x7f7f140ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.563+0000 7f7f1a1ff640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f140d0a00 0x7f7f140ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.563+0000 7f7f1a1ff640 1 -- 172.21.15.97:0/1368844609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1409d240 con 0x7f7f1409ba40 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.563+0000 7f7f1aa00640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f140d0a00 0x7f7f140ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.563+0000 7f7f1a1ff640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f1409ba40 0x7f7f1409cbd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f10000c00 tx=0x7f7f100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.563+0000 7f7f0affd640 1 -- 172.21.15.97:0/1368844609 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10035040 con 0x7f7f1409ba40 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.563+0000 7f7f0affd640 1 -- 172.21.15.97:0/1368844609 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f10030d20 con 0x7f7f1409ba40 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.563+0000 7f7f1c463640 1 -- 172.21.15.97:0/1368844609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1409d520 con 0x7f7f1409ba40 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.563+0000 7f7f0affd640 1 -- 172.21.15.97:0/1368844609 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10044650 con 0x7f7f1409ba40 2024-03-29T04:07:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.563+0000 7f7f1c463640 1 -- 172.21.15.97:0/1368844609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f140c9870 con 0x7f7f1409ba40 2024-03-29T04:07:49.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.564+0000 7f7f1c463640 1 -- 172.21.15.97:0/1368844609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f14068640 con 0x7f7f1409ba40 2024-03-29T04:07:49.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.564+0000 7f7f0affd640 1 -- 172.21.15.97:0/1368844609 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7f10038020 con 0x7f7f1409ba40 2024-03-29T04:07:49.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.565+0000 7f7f0affd640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ef806c530 0x7f7ef806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.565+0000 7f7f0affd640 1 -- 172.21.15.97:0/1368844609 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7f100ad520 con 0x7f7f1409ba40 2024-03-29T04:07:49.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.565+0000 7f7f199fe640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ef806c530 0x7f7ef806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.565+0000 7f7f199fe640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ef806c530 0x7f7ef806e9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f1415b9b0 tx=0x7f7f04007d70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:49.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.569+0000 7f7f0affd640 1 -- 172.21.15.97:0/1368844609 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7f1002d040 con 0x7f7f1409ba40 2024-03-29T04:07:49.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.743+0000 7f7f1c463640 1 -- 172.21.15.97:0/1368844609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f140cc7e0 con 0x7f7f1409ba40 2024-03-29T04:07:49.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.744+0000 7f7f0affd640 1 -- 172.21.15.97:0/1368844609 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f7f1007bbd0 con 0x7f7f1409ba40 2024-03-29T04:07:49.725 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:49.725 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:49.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.746+0000 7f7f1c463640 1 -- 172.21.15.97:0/1368844609 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ef806c530 msgr2=0x7f7ef806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.746+0000 7f7f1c463640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ef806c530 0x7f7ef806e9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f1415b9b0 tx=0x7f7f04007d70 comp rx=0 tx=0).stop 2024-03-29T04:07:49.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.746+0000 7f7f1c463640 1 -- 172.21.15.97:0/1368844609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f1409ba40 msgr2=0x7f7f1409cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.746+0000 7f7f1c463640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f1409ba40 0x7f7f1409cbd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f10000c00 tx=0x7f7f100046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.746+0000 7f7f1c463640 1 -- 172.21.15.97:0/1368844609 shutdown_connections 2024-03-29T04:07:49.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.746+0000 7f7f1c463640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ef806c530 0x7f7ef806e9f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.746+0000 7f7f1c463640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f140d0a00 0x7f7f140ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.746+0000 7f7f1c463640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f140cfff0 0x7f7f140cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.746+0000 7f7f1c463640 1 --2- 172.21.15.97:0/1368844609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f1409ba40 0x7f7f1409cbd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.746+0000 7f7f1c463640 1 -- 172.21.15.97:0/1368844609 >> 172.21.15.97:0/1368844609 conn(0x7f7f140b3f10 msgr2=0x7f7f1415bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:49.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.747+0000 7f7f1c463640 1 -- 172.21.15.97:0/1368844609 shutdown_connections 2024-03-29T04:07:49.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.747+0000 7f7f1c463640 1 -- 172.21.15.97:0/1368844609 wait complete. 2024-03-29T04:07:49.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:07:38.618704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:07:39.724339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:07:49.738 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:07:49.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.861+0000 7fe16aca7640 1 Processor -- start 2024-03-29T04:07:49.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.861+0000 7fe16aca7640 1 -- start start 2024-03-29T04:07:49.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe16aca7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe16409aa30 0x7fe16409b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe16aca7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe16415e130 0x7fe16415e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe16aca7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1640b1bc0 con 0x7fe16409ae10 2024-03-29T04:07:49.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe16aca7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe16406d880 con 0x7fe16415e130 2024-03-29T04:07:49.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe16aca7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fe1640d12a0 con 0x7fe16409aa30 2024-03-29T04:07:49.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe169244640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe16415e130 0x7fe16415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe169244640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe16415e130 0x7fe16415e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39176/0 (socket says 172.21.15.97:39176) 2024-03-29T04:07:49.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe163fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe16409ae10 0x7fe16406d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50494/0 (socket says 172.21.15.97:50494) 2024-03-29T04:07:49.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe169244640 1 -- 172.21.15.97:0/3638184442 learned_addr learned my addr 172.21.15.97:0/3638184442 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:49.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe168a43640 1 --2- 172.21.15.97:0/3638184442 >> v2:172.21.15.154:3300/0 conn(0x7fe16409aa30 0x7fe16409b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe169244640 1 -- 172.21.15.97:0/3638184442 >> v2:172.21.15.154:3300/0 conn(0x7fe16409aa30 msgr2=0x7fe16409b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:49.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe169244640 1 --2- 172.21.15.97:0/3638184442 >> v2:172.21.15.154:3300/0 conn(0x7fe16409aa30 0x7fe16409b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe169244640 1 -- 172.21.15.97:0/3638184442 >> v1:172.21.15.97:6789/0 conn(0x7fe16409ae10 legacy=0x7fe16406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe169244640 1 -- 172.21.15.97:0/3638184442 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1640d14f0 con 0x7fe16415e130 2024-03-29T04:07:49.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.862+0000 7fe1637fe640 1 -- 172.21.15.97:0/3638184442 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2616237545 0 0) 0x7fe1640b1bc0 con 0x7fe16409ae10 2024-03-29T04:07:49.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.863+0000 7fe169244640 1 --2- 172.21.15.97:0/3638184442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16415e130 0x7fe16415e580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe1640723b0 tx=0x7fe15c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d45a5a1b9e80fb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:49.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.863+0000 7fe1637fe640 1 -- 172.21.15.97:0/3638184442 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c02cb20 con 0x7fe16415e130 2024-03-29T04:07:49.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.863+0000 7fe1637fe640 1 -- 172.21.15.97:0/3638184442 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe15c02ccc0 con 0x7fe16415e130 2024-03-29T04:07:49.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.863+0000 7fe1637fe640 1 -- 172.21.15.97:0/3638184442 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c034610 con 0x7fe16415e130 2024-03-29T04:07:49.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.864+0000 7fe16aca7640 1 -- 172.21.15.97:0/3638184442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16415e130 msgr2=0x7fe16415e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.864+0000 7fe16aca7640 1 --2- 172.21.15.97:0/3638184442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16415e130 0x7fe16415e580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe1640723b0 tx=0x7fe15c02bf80 comp rx=0 tx=0).stop 2024-03-29T04:07:49.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.864+0000 7fe16aca7640 1 -- 172.21.15.97:0/3638184442 shutdown_connections 2024-03-29T04:07:49.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.864+0000 7fe16aca7640 1 --2- 172.21.15.97:0/3638184442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16415e130 0x7fe16415e580 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.864+0000 7fe16aca7640 1 --2- 172.21.15.97:0/3638184442 >> v2:172.21.15.154:3300/0 conn(0x7fe16409aa30 0x7fe16409b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.864+0000 7fe16aca7640 1 -- 172.21.15.97:0/3638184442 >> 172.21.15.97:0/3638184442 conn(0x7fe1640d24c0 msgr2=0x7fe1640d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:49.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.864+0000 7fe16aca7640 1 -- 172.21.15.97:0/3638184442 shutdown_connections 2024-03-29T04:07:49.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.864+0000 7fe16aca7640 1 -- 172.21.15.97:0/3638184442 wait complete. 2024-03-29T04:07:49.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.865+0000 7fe16aca7640 1 Processor -- start 2024-03-29T04:07:49.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.865+0000 7fe16aca7640 1 -- start start 2024-03-29T04:07:49.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe16aca7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe16409aa30 0x7fe16414d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe16aca7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16409ae10 0x7fe1640c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe16aca7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe16415e130 0x7fe1640b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe16aca7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe1640b1e20 con 0x7fe16409ae10 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe16aca7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1640d1ab0 con 0x7fe16415e130 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe16aca7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe16409b2a0 con 0x7fe16409aa30 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe163fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16409ae10 0x7fe1640c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe163fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16409ae10 0x7fe1640c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39188/0 (socket says 172.21.15.97:39188) 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe163fff640 1 -- 172.21.15.97:0/2653326030 learned_addr learned my addr 172.21.15.97:0/2653326030 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe168a43640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe16409aa30 0x7fe16414d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe169244640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe16415e130 0x7fe1640b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe163fff640 1 -- 172.21.15.97:0/2653326030 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe16409aa30 msgr2=0x7fe16414d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe163fff640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe16409aa30 0x7fe16414d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe163fff640 1 -- 172.21.15.97:0/2653326030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe16415e130 msgr2=0x7fe1640b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe163fff640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe16415e130 0x7fe1640b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:49.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe163fff640 1 -- 172.21.15.97:0/2653326030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1640c6190 con 0x7fe16409ae10 2024-03-29T04:07:49.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe163fff640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16409ae10 0x7fe1640c6f50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe15000c430 tx=0x7fe15000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:49.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe1617fa640 1 -- 172.21.15.97:0/2653326030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1500074e0 con 0x7fe16409ae10 2024-03-29T04:07:49.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe1617fa640 1 -- 172.21.15.97:0/2653326030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1500047e0 con 0x7fe16409ae10 2024-03-29T04:07:49.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe1617fa640 1 -- 172.21.15.97:0/2653326030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150004bd0 con 0x7fe16409ae10 2024-03-29T04:07:49.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe16aca7640 1 -- 172.21.15.97:0/2653326030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1640c6470 con 0x7fe16409ae10 2024-03-29T04:07:49.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.866+0000 7fe16aca7640 1 -- 172.21.15.97:0/2653326030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1640c7490 con 0x7fe16409ae10 2024-03-29T04:07:49.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.868+0000 7fe16aca7640 1 -- 172.21.15.97:0/2653326030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe164070880 con 0x7fe16409ae10 2024-03-29T04:07:49.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.868+0000 7fe1617fa640 1 -- 172.21.15.97:0/2653326030 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe150011070 con 0x7fe16409ae10 2024-03-29T04:07:49.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.868+0000 7fe1617fa640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe14806c530 0x7fe14806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:49.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.868+0000 7fe1617fa640 1 -- 172.21.15.97:0/2653326030 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe15008b190 con 0x7fe16409ae10 2024-03-29T04:07:49.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.868+0000 7fe168a43640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe14806c530 0x7fe14806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:49.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.869+0000 7fe168a43640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe14806c530 0x7fe14806e9f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe158009aa0 tx=0x7fe158009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:49.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:49.873+0000 7fe1617fa640 1 -- 172.21.15.97:0/2653326030 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe150059160 con 0x7fe16409ae10 2024-03-29T04:07:50.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.044+0000 7fe16aca7640 1 -- 172.21.15.97:0/2653326030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe16404f9d0 con 0x7fe16409ae10 2024-03-29T04:07:50.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.045+0000 7fe1617fa640 1 -- 172.21.15.97:0/2653326030 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fe15005d460 con 0x7fe16409ae10 2024-03-29T04:07:50.025 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:07:50.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.047+0000 7fe16aca7640 1 -- 172.21.15.97:0/2653326030 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe14806c530 msgr2=0x7fe14806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.047+0000 7fe16aca7640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe14806c530 0x7fe14806e9f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe158009aa0 tx=0x7fe158009190 comp rx=0 tx=0).stop 2024-03-29T04:07:50.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.047+0000 7fe16aca7640 1 -- 172.21.15.97:0/2653326030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16409ae10 msgr2=0x7fe1640c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.047+0000 7fe16aca7640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16409ae10 0x7fe1640c6f50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe15000c430 tx=0x7fe15000c8f0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.047+0000 7fe16aca7640 1 -- 172.21.15.97:0/2653326030 shutdown_connections 2024-03-29T04:07:50.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.047+0000 7fe16aca7640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe14806c530 0x7fe14806e9f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.047+0000 7fe16aca7640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe16415e130 0x7fe1640b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.047+0000 7fe16aca7640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe16409ae10 0x7fe1640c6f50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.047+0000 7fe16aca7640 1 --2- 172.21.15.97:0/2653326030 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe16409aa30 0x7fe16414d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.047+0000 7fe16aca7640 1 -- 172.21.15.97:0/2653326030 >> 172.21.15.97:0/2653326030 conn(0x7fe1640d24c0 msgr2=0x7fe16415d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:50.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.047+0000 7fe16aca7640 1 -- 172.21.15.97:0/2653326030 shutdown_connections 2024-03-29T04:07:50.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.047+0000 7fe16aca7640 1 -- 172.21.15.97:0/2653326030 wait complete. 2024-03-29T04:07:50.037 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:07:50.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.160+0000 7fa946190640 1 Processor -- start 2024-03-29T04:07:50.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.160+0000 7fa946190640 1 -- start start 2024-03-29T04:07:50.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa946190640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa9400ae320 0x7fa94015e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa946190640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa94009aa30 0x7fa94015eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa946190640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9400b3f30 con 0x7fa94006b510 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa946190640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa9400b3a00 con 0x7fa9400ae320 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa946190640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa94015e4f0 con 0x7fa94009aa30 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa93effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa9400ae320 0x7fa94015e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa93effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa9400ae320 0x7fa94015e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39204/0 (socket says 172.21.15.97:39204) 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa93f7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa94006b510 0x7fa94006b8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:50496/0 (socket says 172.21.15.97:50496) 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa93effd640 1 -- 172.21.15.97:0/93468772 learned_addr learned my addr 172.21.15.97:0/93468772 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa93ffff640 1 --2- 172.21.15.97:0/93468772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa94009aa30 0x7fa94015eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa93effd640 1 -- 172.21.15.97:0/93468772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa94009aa30 msgr2=0x7fa94015eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa93effd640 1 --2- 172.21.15.97:0/93468772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa94009aa30 0x7fa94015eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa93effd640 1 -- 172.21.15.97:0/93468772 >> v1:172.21.15.97:6789/0 conn(0x7fa94006b510 legacy=0x7fa94006b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa93effd640 1 -- 172.21.15.97:0/93468772 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9400b3c50 con 0x7fa9400ae320 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.161+0000 7fa93e7fc640 1 -- 172.21.15.97:0/93468772 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2952370948 0 0) 0x7fa9400b3f30 con 0x7fa94006b510 2024-03-29T04:07:50.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.162+0000 7fa93effd640 1 --2- 172.21.15.97:0/93468772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9400ae320 0x7fa94015e980 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa934009870 tx=0x7fa93402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d244aae5eb5080a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:50.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.162+0000 7fa93e7fc640 1 -- 172.21.15.97:0/93468772 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934034070 con 0x7fa9400ae320 2024-03-29T04:07:50.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.162+0000 7fa93e7fc640 1 -- 172.21.15.97:0/93468772 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa934030840 con 0x7fa9400ae320 2024-03-29T04:07:50.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.162+0000 7fa93e7fc640 1 -- 172.21.15.97:0/93468772 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934030c10 con 0x7fa9400ae320 2024-03-29T04:07:50.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.163+0000 7fa946190640 1 -- 172.21.15.97:0/93468772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9400ae320 msgr2=0x7fa94015e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.163+0000 7fa946190640 1 --2- 172.21.15.97:0/93468772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9400ae320 0x7fa94015e980 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa934009870 tx=0x7fa93402b980 comp rx=0 tx=0).stop 2024-03-29T04:07:50.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.163+0000 7fa946190640 1 -- 172.21.15.97:0/93468772 shutdown_connections 2024-03-29T04:07:50.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.163+0000 7fa946190640 1 --2- 172.21.15.97:0/93468772 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa94009aa30 0x7fa94015eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.163+0000 7fa946190640 1 --2- 172.21.15.97:0/93468772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9400ae320 0x7fa94015e980 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.163+0000 7fa946190640 1 -- 172.21.15.97:0/93468772 >> 172.21.15.97:0/93468772 conn(0x7fa940072e40 msgr2=0x7fa9400b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:50.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.163+0000 7fa946190640 1 -- 172.21.15.97:0/93468772 shutdown_connections 2024-03-29T04:07:50.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.163+0000 7fa946190640 1 -- 172.21.15.97:0/93468772 wait complete. 2024-03-29T04:07:50.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.164+0000 7fa946190640 1 Processor -- start 2024-03-29T04:07:50.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.164+0000 7fa946190640 1 -- start start 2024-03-29T04:07:50.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa946190640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa94006b510 0x7fa94009c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa946190640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa94009aa30 0x7fa9401f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa946190640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9400ae320 0x7fa9400cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa946190640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa9400b67c0 con 0x7fa9400ae320 2024-03-29T04:07:50.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa946190640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa94015e180 con 0x7fa94009aa30 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa946190640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa940145660 con 0x7fa94006b510 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa93ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9400ae320 0x7fa9400cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa93ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9400ae320 0x7fa9400cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39208/0 (socket says 172.21.15.97:39208) 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa93ffff640 1 -- 172.21.15.97:0/1672700213 learned_addr learned my addr 172.21.15.97:0/1672700213 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa93f7fe640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa94006b510 0x7fa94009c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa93effd640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa94009aa30 0x7fa9401f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa93ffff640 1 -- 172.21.15.97:0/1672700213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa94006b510 msgr2=0x7fa94009c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa93ffff640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa94006b510 0x7fa94009c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa93ffff640 1 -- 172.21.15.97:0/1672700213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa94009aa30 msgr2=0x7fa9401f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa93ffff640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa94009aa30 0x7fa9401f2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa93ffff640 1 -- 172.21.15.97:0/1672700213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9401f3040 con 0x7fa9400ae320 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa93effd640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa94009aa30 0x7fa9401f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa93ffff640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9400ae320 0x7fa9400cc9d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa93800efc0 tx=0x7fa93800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa927fff640 1 -- 172.21.15.97:0/1672700213 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938018020 con 0x7fa9400ae320 2024-03-29T04:07:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa927fff640 1 -- 172.21.15.97:0/1672700213 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa938014480 con 0x7fa9400ae320 2024-03-29T04:07:50.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa946190640 1 -- 172.21.15.97:0/1672700213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94009b680 con 0x7fa9400ae320 2024-03-29T04:07:50.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa927fff640 1 -- 172.21.15.97:0/1672700213 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938014870 con 0x7fa9400ae320 2024-03-29T04:07:50.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.165+0000 7fa946190640 1 -- 172.21.15.97:0/1672700213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94009bc40 con 0x7fa9400ae320 2024-03-29T04:07:50.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.166+0000 7fa946190640 1 -- 172.21.15.97:0/1672700213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94009f6a0 con 0x7fa9400ae320 2024-03-29T04:07:50.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.167+0000 7fa927fff640 1 -- 172.21.15.97:0/1672700213 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa938005d10 con 0x7fa9400ae320 2024-03-29T04:07:50.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.167+0000 7fa927fff640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa92c06c530 0x7fa92c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.167+0000 7fa927fff640 1 -- 172.21.15.97:0/1672700213 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa93808c150 con 0x7fa9400ae320 2024-03-29T04:07:50.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.167+0000 7fa93f7fe640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa92c06c530 0x7fa92c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.168+0000 7fa93f7fe640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa92c06c530 0x7fa92c06e9f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa930000f60 tx=0x7fa930007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:50.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.171+0000 7fa927fff640 1 -- 172.21.15.97:0/1672700213 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa938022070 con 0x7fa9400ae320 2024-03-29T04:07:50.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.346+0000 7fa946190640 1 -- 172.21.15.97:0/1672700213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa94004f9d0 con 0x7fa9400ae320 2024-03-29T04:07:50.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.346+0000 7fa927fff640 1 -- 172.21.15.97:0/1672700213 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa9401f3040 con 0x7fa9400ae320 2024-03-29T04:07:50.326 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:07:50.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.348+0000 7fa946190640 1 -- 172.21.15.97:0/1672700213 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa92c06c530 msgr2=0x7fa92c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.348+0000 7fa946190640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa92c06c530 0x7fa92c06e9f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa930000f60 tx=0x7fa930007d50 comp rx=0 tx=0).stop 2024-03-29T04:07:50.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.348+0000 7fa946190640 1 -- 172.21.15.97:0/1672700213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9400ae320 msgr2=0x7fa9400cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.348+0000 7fa946190640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9400ae320 0x7fa9400cc9d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa93800efc0 tx=0x7fa93800c490 comp rx=0 tx=0).stop 2024-03-29T04:07:50.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.348+0000 7fa946190640 1 -- 172.21.15.97:0/1672700213 shutdown_connections 2024-03-29T04:07:50.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.348+0000 7fa946190640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa92c06c530 0x7fa92c06e9f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.348+0000 7fa946190640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9400ae320 0x7fa9400cc9d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.348+0000 7fa946190640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa94009aa30 0x7fa9401f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.348+0000 7fa946190640 1 --2- 172.21.15.97:0/1672700213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa94006b510 0x7fa94009c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.348+0000 7fa946190640 1 -- 172.21.15.97:0/1672700213 >> 172.21.15.97:0/1672700213 conn(0x7fa940072e40 msgr2=0x7fa940143810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:50.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.348+0000 7fa946190640 1 -- 172.21.15.97:0/1672700213 shutdown_connections 2024-03-29T04:07:50.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.348+0000 7fa946190640 1 -- 172.21.15.97:0/1672700213 wait complete. 2024-03-29T04:07:50.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:07:50.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.463+0000 7f3672952640 1 Processor -- start 2024-03-29T04:07:50.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.463+0000 7f3672952640 1 -- start start 2024-03-29T04:07:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.464+0000 7f3672952640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f366c0b3c50 0x7f366c06c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.464+0000 7f3672952640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f366c0d2c40 0x7f366c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.464+0000 7f3672952640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f366c0d3120 con 0x7f366c06d170 2024-03-29T04:07:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.464+0000 7f3672952640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f366c0d3400 con 0x7f366c0d2c40 2024-03-29T04:07:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.464+0000 7f3672952640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f366c0d36e0 con 0x7f366c0b3c50 2024-03-29T04:07:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.464+0000 7f366b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f366c0d2c40 0x7f366c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.464+0000 7f366b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f366c0d2c40 0x7f366c06cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39224/0 (socket says 172.21.15.97:39224) 2024-03-29T04:07:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.464+0000 7f366b7fe640 1 -- 172.21.15.97:0/1282453871 learned_addr learned my addr 172.21.15.97:0/1282453871 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.464+0000 7f366bfff640 1 --2- 172.21.15.97:0/1282453871 >> v2:172.21.15.154:3300/0 conn(0x7f366c0b3c50 0x7f366c06c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.464+0000 7f366b7fe640 1 -- 172.21.15.97:0/1282453871 >> v2:172.21.15.154:3300/0 conn(0x7f366c0b3c50 msgr2=0x7f366c06c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.464+0000 7f366b7fe640 1 --2- 172.21.15.97:0/1282453871 >> v2:172.21.15.154:3300/0 conn(0x7f366c0b3c50 0x7f366c06c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.464+0000 7f366b7fe640 1 -- 172.21.15.97:0/1282453871 >> v1:172.21.15.154:6789/0 conn(0x7f366c06d170 legacy=0x7f366c1623d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.464+0000 7f366b7fe640 1 -- 172.21.15.97:0/1282453871 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c0cf760 con 0x7f366c0d2c40 2024-03-29T04:07:50.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.465+0000 7f366b7fe640 1 --2- 172.21.15.97:0/1282453871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0d2c40 0x7f366c06cc30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f366c08cf40 tx=0x7f366002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96b2b9d504ead5a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:50.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.465+0000 7f366affd640 1 -- 172.21.15.97:0/1282453871 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660035020 con 0x7f366c0d2c40 2024-03-29T04:07:50.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.465+0000 7f366affd640 1 -- 172.21.15.97:0/1282453871 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3660031870 con 0x7f366c0d2c40 2024-03-29T04:07:50.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.465+0000 7f366affd640 1 -- 172.21.15.97:0/1282453871 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660031c40 con 0x7f366c0d2c40 2024-03-29T04:07:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.466+0000 7f3672952640 1 -- 172.21.15.97:0/1282453871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0d2c40 msgr2=0x7f366c06cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.466+0000 7f3672952640 1 --2- 172.21.15.97:0/1282453871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0d2c40 0x7f366c06cc30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f366c08cf40 tx=0x7f366002b910 comp rx=0 tx=0).stop 2024-03-29T04:07:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.466+0000 7f3672952640 1 -- 172.21.15.97:0/1282453871 shutdown_connections 2024-03-29T04:07:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.466+0000 7f3672952640 1 --2- 172.21.15.97:0/1282453871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0d2c40 0x7f366c06cc30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.466+0000 7f3672952640 1 --2- 172.21.15.97:0/1282453871 >> v2:172.21.15.154:3300/0 conn(0x7f366c0b3c50 0x7f366c06c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.466+0000 7f3672952640 1 -- 172.21.15.97:0/1282453871 >> 172.21.15.97:0/1282453871 conn(0x7f366c08bfc0 msgr2=0x7f366c0d4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.466+0000 7f3672952640 1 -- 172.21.15.97:0/1282453871 shutdown_connections 2024-03-29T04:07:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.466+0000 7f3672952640 1 -- 172.21.15.97:0/1282453871 wait complete. 2024-03-29T04:07:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.467+0000 7f3672952640 1 Processor -- start 2024-03-29T04:07:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.467+0000 7f3672952640 1 -- start start 2024-03-29T04:07:50.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f3672952640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f366c06d170 0x7f366c0b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f3672952640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0b3c50 0x7f366c15eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f3672952640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f366c0d2c40 0x7f366c15f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f3672952640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f366c0d0c90 con 0x7f366c0b3c50 2024-03-29T04:07:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f3672952640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f366c06dae0 con 0x7f366c0d2c40 2024-03-29T04:07:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f3672952640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f366c0d0e10 con 0x7f366c06d170 2024-03-29T04:07:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f366b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0b3c50 0x7f366c15eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f366b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0b3c50 0x7f366c15eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39238/0 (socket says 172.21.15.97:39238) 2024-03-29T04:07:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f366b7fe640 1 -- 172.21.15.97:0/1902696156 learned_addr learned my addr 172.21.15.97:0/1902696156 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f3670eef640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f366c0d2c40 0x7f366c15f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f366bfff640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f366c06d170 0x7f366c0b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f366b7fe640 1 -- 172.21.15.97:0/1902696156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f366c06d170 msgr2=0x7f366c0b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f366b7fe640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f366c06d170 0x7f366c0b7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f366b7fe640 1 -- 172.21.15.97:0/1902696156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f366c0d2c40 msgr2=0x7f366c15f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f366b7fe640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f366c0d2c40 0x7f366c15f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f366b7fe640 1 -- 172.21.15.97:0/1902696156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c0b6230 con 0x7f366c0b3c50 2024-03-29T04:07:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f366b7fe640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0b3c50 0x7f366c15eb50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3660000c00 tx=0x7f36600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f3668ff9640 1 -- 172.21.15.97:0/1902696156 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660035040 con 0x7f366c0b3c50 2024-03-29T04:07:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f3668ff9640 1 -- 172.21.15.97:0/1902696156 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3660030d00 con 0x7f366c0b3c50 2024-03-29T04:07:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f3672952640 1 -- 172.21.15.97:0/1902696156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c0b6510 con 0x7f366c0b3c50 2024-03-29T04:07:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f3668ff9640 1 -- 172.21.15.97:0/1902696156 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36600456a0 con 0x7f366c0b3c50 2024-03-29T04:07:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.468+0000 7f3672952640 1 -- 172.21.15.97:0/1902696156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c0b6ad0 con 0x7f366c0b3c50 2024-03-29T04:07:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.469+0000 7f3672952640 1 -- 172.21.15.97:0/1902696156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366c04f9d0 con 0x7f366c0b3c50 2024-03-29T04:07:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.470+0000 7f3668ff9640 1 -- 172.21.15.97:0/1902696156 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3660038020 con 0x7f366c0b3c50 2024-03-29T04:07:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.470+0000 7f3668ff9640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f365806c530 0x7f365806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.470+0000 7f3668ff9640 1 -- 172.21.15.97:0/1902696156 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f36600ada60 con 0x7f366c0b3c50 2024-03-29T04:07:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.470+0000 7f366bfff640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f365806c530 0x7f365806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.471+0000 7f366bfff640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f365806c530 0x7f365806e9f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f365c009ad0 tx=0x7f365c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:50.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.475+0000 7f3668ff9640 1 -- 172.21.15.97:0/1902696156 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f366002f320 con 0x7f366c0b3c50 2024-03-29T04:07:50.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.649+0000 7f3672952640 1 -- 172.21.15.97:0/1902696156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f366c0b21a0 con 0x7f366c0b3c50 2024-03-29T04:07:50.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.649+0000 7f3668ff9640 1 -- 172.21.15.97:0/1902696156 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f366007c080 con 0x7f366c0b3c50 2024-03-29T04:07:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:07:50.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.652+0000 7f3672952640 1 -- 172.21.15.97:0/1902696156 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f365806c530 msgr2=0x7f365806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.652+0000 7f3672952640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f365806c530 0x7f365806e9f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f365c009ad0 tx=0x7f365c008040 comp rx=0 tx=0).stop 2024-03-29T04:07:50.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.652+0000 7f3672952640 1 -- 172.21.15.97:0/1902696156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0b3c50 msgr2=0x7f366c15eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.652+0000 7f3672952640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0b3c50 0x7f366c15eb50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3660000c00 tx=0x7f36600046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.652+0000 7f3672952640 1 -- 172.21.15.97:0/1902696156 shutdown_connections 2024-03-29T04:07:50.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.652+0000 7f3672952640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f365806c530 0x7f365806e9f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.652+0000 7f3672952640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f366c0d2c40 0x7f366c15f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.652+0000 7f3672952640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c0b3c50 0x7f366c15eb50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.652+0000 7f3672952640 1 --2- 172.21.15.97:0/1902696156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f366c06d170 0x7f366c0b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.652+0000 7f3672952640 1 -- 172.21.15.97:0/1902696156 >> 172.21.15.97:0/1902696156 conn(0x7f366c08bfc0 msgr2=0x7f366c06c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:50.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.652+0000 7f3672952640 1 -- 172.21.15.97:0/1902696156 shutdown_connections 2024-03-29T04:07:50.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.652+0000 7f3672952640 1 -- 172.21.15.97:0/1902696156 wait complete. 2024-03-29T04:07:50.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:07:50.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.763+0000 7f5d41a60640 1 Processor -- start 2024-03-29T04:07:50.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.763+0000 7f5d41a60640 1 -- start start 2024-03-29T04:07:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d41a60640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5d3c0cfff0 0x7f5d3c09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d41a60640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d3c0d0470 0x7f5d3c09af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d41a60640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d3c15db70 con 0x7f5d3c0d0a00 2024-03-29T04:07:50.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d41a60640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f5d3c0cdf90 con 0x7f5d3c0cfff0 2024-03-29T04:07:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d41a60640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5d3c0cd380 con 0x7f5d3c0d0470 2024-03-29T04:07:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d3affd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5d3c0d0a00 0x7f5d3c0d0de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35630/0 (socket says 172.21.15.97:35630) 2024-03-29T04:07:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d3affd640 1 -- 172.21.15.97:0/3156109447 learned_addr learned my addr 172.21.15.97:0/3156109447 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d3a7fc640 1 --2- 172.21.15.97:0/3156109447 >> v2:172.21.15.154:3300/0 conn(0x7f5d3c0cfff0 0x7f5d3c09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d3b7fe640 1 --2- 172.21.15.97:0/3156109447 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d3c0d0470 0x7f5d3c09af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d39ffb640 1 -- 172.21.15.97:0/3156109447 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1146581042 0 0) 0x7f5d3c15db70 con 0x7f5d3c0d0a00 2024-03-29T04:07:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d39ffb640 1 -- 172.21.15.97:0/3156109447 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d24003520 con 0x7f5d3c0d0a00 2024-03-29T04:07:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d3a7fc640 1 -- 172.21.15.97:0/3156109447 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d3c0d0470 msgr2=0x7f5d3c09af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d3a7fc640 1 --2- 172.21.15.97:0/3156109447 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d3c0d0470 0x7f5d3c09af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.764+0000 7f5d3a7fc640 1 -- 172.21.15.97:0/3156109447 >> v1:172.21.15.97:6789/0 conn(0x7f5d3c0d0a00 legacy=0x7f5d3c0d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.765+0000 7f5d3a7fc640 1 -- 172.21.15.97:0/3156109447 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c0cd660 con 0x7f5d3c0cfff0 2024-03-29T04:07:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.765+0000 7f5d3a7fc640 1 --2- 172.21.15.97:0/3156109447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d3c0cfff0 0x7f5d3c09aa30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5d300097a0 tx=0x7f5d3002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49bc46e428b47c23 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.765+0000 7f5d39ffb640 1 -- 172.21.15.97:0/3156109447 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30035020 con 0x7f5d3c0cfff0 2024-03-29T04:07:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.765+0000 7f5d39ffb640 1 -- 172.21.15.97:0/3156109447 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d30031830 con 0x7f5d3c0cfff0 2024-03-29T04:07:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.765+0000 7f5d39ffb640 1 -- 172.21.15.97:0/3156109447 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30031be0 con 0x7f5d3c0cfff0 2024-03-29T04:07:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.766+0000 7f5d41a60640 1 -- 172.21.15.97:0/3156109447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d3c0cfff0 msgr2=0x7f5d3c09aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.766+0000 7f5d41a60640 1 --2- 172.21.15.97:0/3156109447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d3c0cfff0 0x7f5d3c09aa30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5d300097a0 tx=0x7f5d3002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.766+0000 7f5d41a60640 1 -- 172.21.15.97:0/3156109447 shutdown_connections 2024-03-29T04:07:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.766+0000 7f5d41a60640 1 --2- 172.21.15.97:0/3156109447 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d3c0d0470 0x7f5d3c09af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.766+0000 7f5d41a60640 1 --2- 172.21.15.97:0/3156109447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d3c0cfff0 0x7f5d3c09aa30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.766+0000 7f5d41a60640 1 -- 172.21.15.97:0/3156109447 >> 172.21.15.97:0/3156109447 conn(0x7f5d3c0b3f10 msgr2=0x7f5d3c0b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.766+0000 7f5d41a60640 1 -- 172.21.15.97:0/3156109447 shutdown_connections 2024-03-29T04:07:50.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.766+0000 7f5d41a60640 1 -- 172.21.15.97:0/3156109447 wait complete. 2024-03-29T04:07:50.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.767+0000 7f5d41a60640 1 Processor -- start 2024-03-29T04:07:50.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.767+0000 7f5d41a60640 1 -- start start 2024-03-29T04:07:50.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d41a60640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d3c0cfff0 0x7f5d3c14e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d41a60640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d3c0d0470 0x7f5d3c147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d41a60640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d3c0d0a00 0x7f5d3c14a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d41a60640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c15c960 con 0x7f5d3c0d0470 2024-03-29T04:07:50.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d41a60640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c0cdc20 con 0x7f5d3c0cfff0 2024-03-29T04:07:50.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d41a60640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5d3c0ccf30 con 0x7f5d3c0d0a00 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d3a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d3c0d0470 0x7f5d3c147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d3a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d3c0d0470 0x7f5d3c147df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52054/0 (socket says 172.21.15.97:52054) 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d3a7fc640 1 -- 172.21.15.97:0/535031198 learned_addr learned my addr 172.21.15.97:0/535031198 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d3affd640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d3c0cfff0 0x7f5d3c14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d3b7fe640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d3c0d0a00 0x7f5d3c14a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d3a7fc640 1 -- 172.21.15.97:0/535031198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d3c0d0a00 msgr2=0x7f5d3c14a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d3a7fc640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d3c0d0a00 0x7f5d3c14a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d3a7fc640 1 -- 172.21.15.97:0/535031198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d3c0cfff0 msgr2=0x7f5d3c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d3a7fc640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d3c0cfff0 0x7f5d3c14e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.768+0000 7f5d3a7fc640 1 -- 172.21.15.97:0/535031198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c0b6e00 con 0x7f5d3c0d0470 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.769+0000 7f5d3a7fc640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d3c0d0470 0x7f5d3c147df0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d300098c0 tx=0x7f5d300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.769+0000 7f5d23fff640 1 -- 172.21.15.97:0/535031198 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30035040 con 0x7f5d3c0d0470 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.769+0000 7f5d23fff640 1 -- 172.21.15.97:0/535031198 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d3003b900 con 0x7f5d3c0d0470 2024-03-29T04:07:50.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.769+0000 7f5d41a60640 1 -- 172.21.15.97:0/535031198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c0b70e0 con 0x7f5d3c0d0470 2024-03-29T04:07:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.769+0000 7f5d23fff640 1 -- 172.21.15.97:0/535031198 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3003bcf0 con 0x7f5d3c0d0470 2024-03-29T04:07:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.769+0000 7f5d41a60640 1 -- 172.21.15.97:0/535031198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c0b76a0 con 0x7f5d3c0d0470 2024-03-29T04:07:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.770+0000 7f5d41a60640 1 -- 172.21.15.97:0/535031198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c068640 con 0x7f5d3c0d0470 2024-03-29T04:07:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.770+0000 7f5d23fff640 1 -- 172.21.15.97:0/535031198 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5d30038020 con 0x7f5d3c0d0470 2024-03-29T04:07:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.771+0000 7f5d23fff640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5d2806c530 0x7f5d2806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.771+0000 7f5d23fff640 1 -- 172.21.15.97:0/535031198 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5d300ad610 con 0x7f5d3c0d0470 2024-03-29T04:07:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.771+0000 7f5d3affd640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5d2806c530 0x7f5d2806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:50.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.771+0000 7f5d3affd640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5d2806c530 0x7f5d2806e9f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005c00 tx=0x7f5d2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:50.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.775+0000 7f5d23fff640 1 -- 172.21.15.97:0/535031198 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5d3007b5e0 con 0x7f5d3c0d0470 2024-03-29T04:07:50.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.990+0000 7f5d41a60640 1 -- 172.21.15.97:0/535031198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d3c0b4b00 con 0x7f5d3c0d0470 2024-03-29T04:07:50.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.990+0000 7f5d23fff640 1 -- 172.21.15.97:0/535031198 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7f5d3007f8e0 con 0x7f5d3c0d0470 2024-03-29T04:07:50.970 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:50.970 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1127043830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127043830},{"type":"v1","addr":"172.21.15.154:6834","nonce":1127043830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3767188750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3767188750},{"type":"v1","addr":"172.21.15.97:6835","nonce":3767188750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3651340478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3651340478},{"type":"v1","addr":"172.21.15.97:6837","nonce":3651340478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:07:40.761466+0000","modified":"2024-03-29T04:07:42.771721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3669418265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3669418265},{"type":"v1","addr":"172.21.15.154:6835","nonce":3669418265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-29T04:07:50.971 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-03-29T04:07:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.992+0000 7f5d41a60640 1 -- 172.21.15.97:0/535031198 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5d2806c530 msgr2=0x7f5d2806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.992+0000 7f5d41a60640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5d2806c530 0x7f5d2806e9f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005c00 tx=0x7f5d2c009040 comp rx=0 tx=0).stop 2024-03-29T04:07:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.992+0000 7f5d41a60640 1 -- 172.21.15.97:0/535031198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d3c0d0470 msgr2=0x7f5d3c147df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.992+0000 7f5d41a60640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d3c0d0470 0x7f5d3c147df0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d300098c0 tx=0x7f5d300046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.993+0000 7f5d41a60640 1 -- 172.21.15.97:0/535031198 shutdown_connections 2024-03-29T04:07:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.993+0000 7f5d41a60640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5d2806c530 0x7f5d2806e9f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.993+0000 7f5d41a60640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d3c0d0a00 0x7f5d3c14a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.993+0000 7f5d41a60640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d3c0d0470 0x7f5d3c147df0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.993+0000 7f5d41a60640 1 --2- 172.21.15.97:0/535031198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d3c0cfff0 0x7f5d3c14e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.993+0000 7f5d41a60640 1 -- 172.21.15.97:0/535031198 >> 172.21.15.97:0/535031198 conn(0x7f5d3c0b3f10 msgr2=0x7f5d3c15ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.993+0000 7f5d41a60640 1 -- 172.21.15.97:0/535031198 shutdown_connections 2024-03-29T04:07:50.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:50.993+0000 7f5d41a60640 1 -- 172.21.15.97:0/535031198 wait complete. 2024-03-29T04:07:50.983 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:51.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.103+0000 7f37097af640 1 Processor -- start 2024-03-29T04:07:51.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.103+0000 7f37097af640 1 -- start start 2024-03-29T04:07:51.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.104+0000 7f37097af640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f37040ae1b0 0x7f37040ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.104+0000 7f37097af640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37040b5dc0 con 0x7f37040b0930 2024-03-29T04:07:51.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.104+0000 7f37097af640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37040b6040 con 0x7f370415e9a0 2024-03-29T04:07:51.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.104+0000 7f37097af640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f37040b6320 con 0x7f37040ae1b0 2024-03-29T04:07:51.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.104+0000 7f37037fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f37040ae1b0 0x7f37040ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.104+0000 7f37027fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f37040b0930 0x7f370406f4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35634/0 (socket says 172.21.15.97:35634) 2024-03-29T04:07:51.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.104+0000 7f37027fc640 1 -- 172.21.15.97:0/901373534 learned_addr learned my addr 172.21.15.97:0/901373534 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:51.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.105+0000 7f37037fe640 1 -- 172.21.15.97:0/901373534 >> v1:172.21.15.154:6789/0 conn(0x7f370415e9a0 legacy=0x7f3704070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.105+0000 7f37037fe640 1 -- 172.21.15.97:0/901373534 >> v1:172.21.15.97:6789/0 conn(0x7f37040b0930 legacy=0x7f370406f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.105+0000 7f3701ffb640 1 -- 172.21.15.97:0/901373534 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1357366834 0 0) 0x7f36ec001110 con 0x7f37040b0930 2024-03-29T04:07:51.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.105+0000 7f37037fe640 1 -- 172.21.15.97:0/901373534 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37040b43c0 con 0x7f37040ae1b0 2024-03-29T04:07:51.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.105+0000 7f37037fe640 1 --2- 172.21.15.97:0/901373534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37040ae1b0 0x7f37040ae600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0097a0 tx=0x7f36fc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f825914ef43b3e82 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:51.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.105+0000 7f3701ffb640 1 -- 172.21.15.97:0/901373534 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc034070 con 0x7f37040ae1b0 2024-03-29T04:07:51.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.105+0000 7f3701ffb640 1 -- 172.21.15.97:0/901373534 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36fc030610 con 0x7f37040ae1b0 2024-03-29T04:07:51.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.106+0000 7f3701ffb640 1 -- 172.21.15.97:0/901373534 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0309e0 con 0x7f37040ae1b0 2024-03-29T04:07:51.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.106+0000 7f37097af640 1 -- 172.21.15.97:0/901373534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37040ae1b0 msgr2=0x7f37040ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.106+0000 7f37097af640 1 --2- 172.21.15.97:0/901373534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37040ae1b0 0x7f37040ae600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0097a0 tx=0x7f36fc02b960 comp rx=0 tx=0).stop 2024-03-29T04:07:51.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.106+0000 7f37097af640 1 -- 172.21.15.97:0/901373534 shutdown_connections 2024-03-29T04:07:51.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.106+0000 7f37097af640 1 --2- 172.21.15.97:0/901373534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37040ae1b0 0x7f37040ae600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.106+0000 7f37097af640 1 -- 172.21.15.97:0/901373534 >> 172.21.15.97:0/901373534 conn(0x7f370409aa30 msgr2=0x7f37040b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:51.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.106+0000 7f37097af640 1 -- 172.21.15.97:0/901373534 shutdown_connections 2024-03-29T04:07:51.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.106+0000 7f37097af640 1 -- 172.21.15.97:0/901373534 wait complete. 2024-03-29T04:07:51.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.107+0000 7f37097af640 1 Processor -- start 2024-03-29T04:07:51.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.107+0000 7f37097af640 1 -- start start 2024-03-29T04:07:51.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37097af640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37040ae1b0 0x7f37041e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37097af640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37040b0930 0x7f37041489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37097af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f370415e9a0 0x7f3704160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37097af640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f370406fd40 con 0x7f370415e9a0 2024-03-29T04:07:51.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37097af640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f370406fbc0 con 0x7f37040ae1b0 2024-03-29T04:07:51.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37097af640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f370406b510 con 0x7f37040b0930 2024-03-29T04:07:51.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37037fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f370415e9a0 0x7f3704160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37037fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f370415e9a0 0x7f3704160f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52074/0 (socket says 172.21.15.97:52074) 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37037fe640 1 -- 172.21.15.97:0/1557321597 learned_addr learned my addr 172.21.15.97:0/1557321597 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37027fc640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37040b0930 0x7f37041489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f3702ffd640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37040ae1b0 0x7f37041e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37037fe640 1 -- 172.21.15.97:0/1557321597 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37040b0930 msgr2=0x7f37041489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37037fe640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37040b0930 0x7f37041489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37037fe640 1 -- 172.21.15.97:0/1557321597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37040ae1b0 msgr2=0x7f37041e7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37037fe640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37040ae1b0 0x7f37041e7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.108+0000 7f37037fe640 1 -- 172.21.15.97:0/1557321597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37041e75c0 con 0x7f370415e9a0 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.109+0000 7f37037fe640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f370415e9a0 0x7f3704160f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0098c0 tx=0x7f36fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.109+0000 7f36fb7fe640 1 -- 172.21.15.97:0/1557321597 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc034040 con 0x7f370415e9a0 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.109+0000 7f36fb7fe640 1 -- 172.21.15.97:0/1557321597 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36fc039990 con 0x7f370415e9a0 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.109+0000 7f37097af640 1 -- 172.21.15.97:0/1557321597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370415fe90 con 0x7f370415e9a0 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.109+0000 7f36fb7fe640 1 -- 172.21.15.97:0/1557321597 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc039d80 con 0x7f370415e9a0 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.109+0000 7f37097af640 1 -- 172.21.15.97:0/1557321597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3704160450 con 0x7f370415e9a0 2024-03-29T04:07:51.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.110+0000 7f37097af640 1 -- 172.21.15.97:0/1557321597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3704068d50 con 0x7f370415e9a0 2024-03-29T04:07:51.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.110+0000 7f36fb7fe640 1 -- 172.21.15.97:0/1557321597 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f36fc037070 con 0x7f370415e9a0 2024-03-29T04:07:51.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.110+0000 7f36fb7fe640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f36e406c530 0x7f36e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.111+0000 7f36fb7fe640 1 -- 172.21.15.97:0/1557321597 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f36fc0ad5d0 con 0x7f370415e9a0 2024-03-29T04:07:51.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.111+0000 7f3702ffd640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f36e406c530 0x7f36e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.111+0000 7f3702ffd640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f36e406c530 0x7f36e406e9f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f36f40012e0 tx=0x7f36f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:51.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.115+0000 7f36fb7fe640 1 -- 172.21.15.97:0/1557321597 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f36fc07b630 con 0x7f370415e9a0 2024-03-29T04:07:51.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.288+0000 7f37097af640 1 -- 172.21.15.97:0/1557321597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f370415d6a0 con 0x7f370415e9a0 2024-03-29T04:07:51.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.289+0000 7f36fb7fe640 1 -- 172.21.15.97:0/1557321597 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f36fc07f930 con 0x7f370415e9a0 2024-03-29T04:07:51.269 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:51.269 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.291+0000 7f37097af640 1 -- 172.21.15.97:0/1557321597 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f36e406c530 msgr2=0x7f36e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.291+0000 7f37097af640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f36e406c530 0x7f36e406e9f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f36f40012e0 tx=0x7f36f4009040 comp rx=0 tx=0).stop 2024-03-29T04:07:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.291+0000 7f37097af640 1 -- 172.21.15.97:0/1557321597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f370415e9a0 msgr2=0x7f3704160f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.291+0000 7f37097af640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f370415e9a0 0x7f3704160f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0098c0 tx=0x7f36fc0046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.291+0000 7f37097af640 1 -- 172.21.15.97:0/1557321597 shutdown_connections 2024-03-29T04:07:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.291+0000 7f37097af640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f36e406c530 0x7f36e406e9f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.291+0000 7f37097af640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f370415e9a0 0x7f3704160f30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.291+0000 7f37097af640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f37040b0930 0x7f37041489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.291+0000 7f37097af640 1 --2- 172.21.15.97:0/1557321597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37040ae1b0 0x7f37041e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.291+0000 7f37097af640 1 -- 172.21.15.97:0/1557321597 >> 172.21.15.97:0/1557321597 conn(0x7f370409aa30 msgr2=0x7f370406ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.291+0000 7f37097af640 1 -- 172.21.15.97:0/1557321597 shutdown_connections 2024-03-29T04:07:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.291+0000 7f37097af640 1 -- 172.21.15.97:0/1557321597 wait complete. 2024-03-29T04:07:51.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:07:38.618704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:07:39.724339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:07:51.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:07:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.405+0000 7f6b2c8fb640 1 Processor -- start 2024-03-29T04:07:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.405+0000 7f6b2c8fb640 1 -- start start 2024-03-29T04:07:51.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.406+0000 7f6b2c8fb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b280cf3f0 0x7f6b280b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.406+0000 7f6b2c8fb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b28160140 con 0x7f6b2815f6e0 2024-03-29T04:07:51.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.406+0000 7f6b2c8fb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b280b1b60 con 0x7f6b280b2a70 2024-03-29T04:07:51.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.406+0000 7f6b2c8fb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f6b280b03c0 con 0x7f6b280cf3f0 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.406+0000 7f6b2659c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6b2815f6e0 0x7f6b280afcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35646/0 (socket says 172.21.15.97:35646) 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.406+0000 7f6b2659c640 1 -- 172.21.15.97:0/1493456149 learned_addr learned my addr 172.21.15.97:0/1493456149 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.407+0000 7f6b26d9d640 1 --2- 172.21.15.97:0/1493456149 >> v2:172.21.15.154:3300/0 conn(0x7f6b280cf3f0 0x7f6b280b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.407+0000 7f6b2559a640 1 -- 172.21.15.97:0/1493456149 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1726044181 0 0) 0x7f6b28160140 con 0x7f6b2815f6e0 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.407+0000 7f6b2559a640 1 -- 172.21.15.97:0/1493456149 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b10003520 con 0x7f6b2815f6e0 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.407+0000 7f6b2559a640 1 -- 172.21.15.97:0/1493456149 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4084070369 0 0) 0x7f6b280b1b60 con 0x7f6b280b2a70 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.407+0000 7f6b2559a640 1 -- 172.21.15.97:0/1493456149 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b28160140 con 0x7f6b280b2a70 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.407+0000 7f6b26d9d640 1 -- 172.21.15.97:0/1493456149 >> v1:172.21.15.154:6789/0 conn(0x7f6b280b2a70 legacy=0x7f6b280ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.407+0000 7f6b26d9d640 1 -- 172.21.15.97:0/1493456149 >> v1:172.21.15.97:6789/0 conn(0x7f6b2815f6e0 legacy=0x7f6b280afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.407+0000 7f6b26d9d640 1 -- 172.21.15.97:0/1493456149 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b280b0540 con 0x7f6b280cf3f0 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.408+0000 7f6b26d9d640 1 --2- 172.21.15.97:0/1493456149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b280cf3f0 0x7f6b280b1e40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009d90 tx=0x7f6b2002e0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de65d1d563d25cd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.408+0000 7f6b2559a640 1 -- 172.21.15.97:0/1493456149 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2003b070 con 0x7f6b280cf3f0 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.408+0000 7f6b2559a640 1 -- 172.21.15.97:0/1493456149 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b20036440 con 0x7f6b280cf3f0 2024-03-29T04:07:51.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.408+0000 7f6b2559a640 1 -- 172.21.15.97:0/1493456149 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20036810 con 0x7f6b280cf3f0 2024-03-29T04:07:51.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.408+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/1493456149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b280cf3f0 msgr2=0x7f6b280b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.408+0000 7f6b2c8fb640 1 --2- 172.21.15.97:0/1493456149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b280cf3f0 0x7f6b280b1e40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009d90 tx=0x7f6b2002e0c0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.408+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/1493456149 shutdown_connections 2024-03-29T04:07:51.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.408+0000 7f6b2c8fb640 1 --2- 172.21.15.97:0/1493456149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b280cf3f0 0x7f6b280b1e40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.408+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/1493456149 >> 172.21.15.97:0/1493456149 conn(0x7f6b280b2ed0 msgr2=0x7f6b280b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:51.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.408+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/1493456149 shutdown_connections 2024-03-29T04:07:51.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.408+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/1493456149 wait complete. 2024-03-29T04:07:51.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.410+0000 7f6b2c8fb640 1 Processor -- start 2024-03-29T04:07:51.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.410+0000 7f6b2c8fb640 1 -- start start 2024-03-29T04:07:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.410+0000 7f6b2c8fb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b280b2a70 0x7f6b28144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.410+0000 7f6b2c8fb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b280cf3f0 0x7f6b280cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.410+0000 7f6b2c8fb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2815f6e0 0x7f6b280c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.410+0000 7f6b2c8fb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2815fed0 con 0x7f6b280b2a70 2024-03-29T04:07:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.410+0000 7f6b2c8fb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2815fd50 con 0x7f6b280cf3f0 2024-03-29T04:07:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.410+0000 7f6b2c8fb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6b280b0790 con 0x7f6b2815f6e0 2024-03-29T04:07:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b2659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b280b2a70 0x7f6b28144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b2659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b280b2a70 0x7f6b28144b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52084/0 (socket says 172.21.15.97:52084) 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b2659c640 1 -- 172.21.15.97:0/4148283712 learned_addr learned my addr 172.21.15.97:0/4148283712 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b25d9b640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b280cf3f0 0x7f6b280cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b26d9d640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2815f6e0 0x7f6b280c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b2659c640 1 -- 172.21.15.97:0/4148283712 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2815f6e0 msgr2=0x7f6b280c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b2659c640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2815f6e0 0x7f6b280c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b2659c640 1 -- 172.21.15.97:0/4148283712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b280cf3f0 msgr2=0x7f6b280cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b2659c640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b280cf3f0 0x7f6b280cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b2659c640 1 -- 172.21.15.97:0/4148283712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2815a230 con 0x7f6b280b2a70 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b25d9b640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b280cf3f0 0x7f6b280cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b2659c640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b280b2a70 0x7f6b28144b80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00c500 tx=0x7f6b1c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b16ffd640 1 -- 172.21.15.97:0/4148283712 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c017070 con 0x7f6b280b2a70 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b16ffd640 1 -- 172.21.15.97:0/4148283712 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1c005d00 con 0x7f6b280b2a70 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/4148283712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2815a510 con 0x7f6b280b2a70 2024-03-29T04:07:51.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b16ffd640 1 -- 172.21.15.97:0/4148283712 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c007330 con 0x7f6b280b2a70 2024-03-29T04:07:51.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.411+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/4148283712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2815aad0 con 0x7f6b280b2a70 2024-03-29T04:07:51.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.412+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/4148283712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2809f530 con 0x7f6b280b2a70 2024-03-29T04:07:51.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.413+0000 7f6b16ffd640 1 -- 172.21.15.97:0/4148283712 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6b1c01a070 con 0x7f6b280b2a70 2024-03-29T04:07:51.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.413+0000 7f6b16ffd640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6b0806c530 0x7f6b0806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.413+0000 7f6b25d9b640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6b0806c530 0x7f6b0806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.413+0000 7f6b16ffd640 1 -- 172.21.15.97:0/4148283712 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6b1c08aa60 con 0x7f6b280b2a70 2024-03-29T04:07:51.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.414+0000 7f6b25d9b640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6b0806c530 0x7f6b0806e9f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6b18007fd0 tx=0x7f6b18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:51.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.418+0000 7f6b16ffd640 1 -- 172.21.15.97:0/4148283712 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6b1c058b80 con 0x7f6b280b2a70 2024-03-29T04:07:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.594+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/4148283712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6b28073400 con 0x7f6b280b2a70 2024-03-29T04:07:51.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.594+0000 7f6b16ffd640 1 -- 172.21.15.97:0/4148283712 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f6b1c05ce80 con 0x7f6b280b2a70 2024-03-29T04:07:51.574 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:07:51.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.596+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/4148283712 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6b0806c530 msgr2=0x7f6b0806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.596+0000 7f6b2c8fb640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6b0806c530 0x7f6b0806e9f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6b18007fd0 tx=0x7f6b18009040 comp rx=0 tx=0).stop 2024-03-29T04:07:51.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.596+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/4148283712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b280b2a70 msgr2=0x7f6b28144b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.596+0000 7f6b2c8fb640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b280b2a70 0x7f6b28144b80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00c500 tx=0x7f6b1c00c9c0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.596+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/4148283712 shutdown_connections 2024-03-29T04:07:51.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.596+0000 7f6b2c8fb640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6b0806c530 0x7f6b0806e9f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.596+0000 7f6b2c8fb640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2815f6e0 0x7f6b280c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.596+0000 7f6b2c8fb640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b280cf3f0 0x7f6b280cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.596+0000 7f6b2c8fb640 1 --2- 172.21.15.97:0/4148283712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b280b2a70 0x7f6b28144b80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.596+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/4148283712 >> 172.21.15.97:0/4148283712 conn(0x7f6b280b2ed0 msgr2=0x7f6b2806e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:51.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.596+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/4148283712 shutdown_connections 2024-03-29T04:07:51.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.596+0000 7f6b2c8fb640 1 -- 172.21.15.97:0/4148283712 wait complete. 2024-03-29T04:07:51.587 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:07:51.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.711+0000 7fbe8f2b1640 1 Processor -- start 2024-03-29T04:07:51.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.711+0000 7fbe8f2b1640 1 -- start start 2024-03-29T04:07:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe8f2b1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbe8806bc20 0x7fbe880b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe8f2b1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe880b6500 con 0x7fbe880b6020 2024-03-29T04:07:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe8f2b1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe880b4470 con 0x7fbe8815e8b0 2024-03-29T04:07:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe8f2b1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fbe880b67e0 con 0x7fbe8806bc20 2024-03-29T04:07:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe8c84c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbe880b6020 0x7fbe8806b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35656/0 (socket says 172.21.15.97:35656) 2024-03-29T04:07:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe8c84c640 1 -- 172.21.15.97:0/4028520340 learned_addr learned my addr 172.21.15.97:0/4028520340 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe8d84e640 1 --2- 172.21.15.97:0/4028520340 >> v2:172.21.15.154:3300/0 conn(0x7fbe8806bc20 0x7fbe880b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe7ffff640 1 -- 172.21.15.97:0/4028520340 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 466043824 0 0) 0x7fbe880b6500 con 0x7fbe880b6020 2024-03-29T04:07:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe7ffff640 1 -- 172.21.15.97:0/4028520340 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe74003520 con 0x7fbe880b6020 2024-03-29T04:07:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe7ffff640 1 -- 172.21.15.97:0/4028520340 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1163197550 0 0) 0x7fbe880b4470 con 0x7fbe8815e8b0 2024-03-29T04:07:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe7ffff640 1 -- 172.21.15.97:0/4028520340 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe880b6500 con 0x7fbe8815e8b0 2024-03-29T04:07:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe8d84e640 1 -- 172.21.15.97:0/4028520340 >> v1:172.21.15.154:6789/0 conn(0x7fbe8815e8b0 legacy=0x7fbe880b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe8d84e640 1 -- 172.21.15.97:0/4028520340 >> v1:172.21.15.97:6789/0 conn(0x7fbe880b6020 legacy=0x7fbe8806b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.712+0000 7fbe8d84e640 1 -- 172.21.15.97:0/4028520340 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe880b4750 con 0x7fbe8806bc20 2024-03-29T04:07:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.713+0000 7fbe8d84e640 1 --2- 172.21.15.97:0/4028520340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8806bc20 0x7fbe880b2a40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009d90 tx=0x7fbe8002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e186e89eba0c12c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.713+0000 7fbe7ffff640 1 -- 172.21.15.97:0/4028520340 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80039070 con 0x7fbe8806bc20 2024-03-29T04:07:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.713+0000 7fbe7ffff640 1 -- 172.21.15.97:0/4028520340 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe8002cc60 con 0x7fbe8806bc20 2024-03-29T04:07:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.713+0000 7fbe7ffff640 1 -- 172.21.15.97:0/4028520340 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80034640 con 0x7fbe8806bc20 2024-03-29T04:07:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.714+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/4028520340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8806bc20 msgr2=0x7fbe880b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.714+0000 7fbe8f2b1640 1 --2- 172.21.15.97:0/4028520340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8806bc20 0x7fbe880b2a40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009d90 tx=0x7fbe8002beb0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.714+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/4028520340 shutdown_connections 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.714+0000 7fbe8f2b1640 1 --2- 172.21.15.97:0/4028520340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8806bc20 0x7fbe880b2a40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.714+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/4028520340 >> 172.21.15.97:0/4028520340 conn(0x7fbe88072e40 msgr2=0x7fbe880b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.714+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/4028520340 shutdown_connections 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.714+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/4028520340 wait complete. 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.715+0000 7fbe8f2b1640 1 Processor -- start 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.715+0000 7fbe8f2b1640 1 -- start start 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8f2b1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8806bc20 0x7fbe8814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8f2b1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe880b6020 0x7fbe8814b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8f2b1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe8815e8b0 0x7fbe8815f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8f2b1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8806c190 con 0x7fbe8815e8b0 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8f2b1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbe880b4b30 con 0x7fbe8806bc20 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8f2b1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbe8806c310 con 0x7fbe880b6020 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8d84e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe8815e8b0 0x7fbe8815f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8d84e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe8815e8b0 0x7fbe8815f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52088/0 (socket says 172.21.15.97:52088) 2024-03-29T04:07:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8d84e640 1 -- 172.21.15.97:0/923637501 learned_addr learned my addr 172.21.15.97:0/923637501 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8d04d640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8806bc20 0x7fbe8814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8c84c640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe880b6020 0x7fbe8814b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8d84e640 1 -- 172.21.15.97:0/923637501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe880b6020 msgr2=0x7fbe8814b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8d84e640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe880b6020 0x7fbe8814b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8d84e640 1 -- 172.21.15.97:0/923637501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8806bc20 msgr2=0x7fbe8814ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8d84e640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8806bc20 0x7fbe8814ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8d84e640 1 -- 172.21.15.97:0/923637501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe881522b0 con 0x7fbe8815e8b0 2024-03-29T04:07:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8d84e640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe8815e8b0 0x7fbe8815f7d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009eb0 tx=0x7fbe80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe7dffb640 1 -- 172.21.15.97:0/923637501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80039050 con 0x7fbe8815e8b0 2024-03-29T04:07:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe7dffb640 1 -- 172.21.15.97:0/923637501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe80045070 con 0x7fbe8815e8b0 2024-03-29T04:07:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe7dffb640 1 -- 172.21.15.97:0/923637501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80009450 con 0x7fbe8815e8b0 2024-03-29T04:07:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.716+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/923637501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe88152590 con 0x7fbe8815e8b0 2024-03-29T04:07:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.717+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/923637501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8814ca60 con 0x7fbe8815e8b0 2024-03-29T04:07:51.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.717+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/923637501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8809f6a0 con 0x7fbe8815e8b0 2024-03-29T04:07:51.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.718+0000 7fbe7dffb640 1 -- 172.21.15.97:0/923637501 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbe8003e830 con 0x7fbe8815e8b0 2024-03-29T04:07:51.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.718+0000 7fbe7dffb640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fbe6c06c530 0x7fbe6c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.718+0000 7fbe8d04d640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fbe6c06c530 0x7fbe6c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.718+0000 7fbe7dffb640 1 -- 172.21.15.97:0/923637501 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbe800ae320 con 0x7fbe8815e8b0 2024-03-29T04:07:51.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.719+0000 7fbe8d04d640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fbe6c06c530 0x7fbe6c06e9f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe84007fd0 tx=0x7fbe84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.723+0000 7fbe7dffb640 1 -- 172.21.15.97:0/923637501 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbe8007c4d0 con 0x7fbe8815e8b0 2024-03-29T04:07:51.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.900+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/923637501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbe8804f9d0 con 0x7fbe8815e8b0 2024-03-29T04:07:51.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.900+0000 7fbe7dffb640 1 -- 172.21.15.97:0/923637501 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fbe800807d0 con 0x7fbe8815e8b0 2024-03-29T04:07:51.880 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:07:51.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.902+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/923637501 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fbe6c06c530 msgr2=0x7fbe6c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.902+0000 7fbe8f2b1640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fbe6c06c530 0x7fbe6c06e9f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe84007fd0 tx=0x7fbe84009040 comp rx=0 tx=0).stop 2024-03-29T04:07:51.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.902+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/923637501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe8815e8b0 msgr2=0x7fbe8815f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.902+0000 7fbe8f2b1640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe8815e8b0 0x7fbe8815f7d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009eb0 tx=0x7fbe80004a40 comp rx=0 tx=0).stop 2024-03-29T04:07:51.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.902+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/923637501 shutdown_connections 2024-03-29T04:07:51.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.902+0000 7fbe8f2b1640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fbe6c06c530 0x7fbe6c06e9f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.902+0000 7fbe8f2b1640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe8815e8b0 0x7fbe8815f7d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.902+0000 7fbe8f2b1640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbe880b6020 0x7fbe8814b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.902+0000 7fbe8f2b1640 1 --2- 172.21.15.97:0/923637501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe8806bc20 0x7fbe8814ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.902+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/923637501 >> 172.21.15.97:0/923637501 conn(0x7fbe88072e40 msgr2=0x7fbe8806b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:51.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.902+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/923637501 shutdown_connections 2024-03-29T04:07:51.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:51.903+0000 7fbe8f2b1640 1 -- 172.21.15.97:0/923637501 wait complete. 2024-03-29T04:07:51.893 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:07:51.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.016+0000 7f00f8715640 1 Processor -- start 2024-03-29T04:07:51.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.016+0000 7f00f8715640 1 -- start start 2024-03-29T04:07:51.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.017+0000 7f00f8715640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00f0072dc0 0x7f00f00cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.017+0000 7f00f8715640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f00f006c520 0x7f00f006c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.017+0000 7f00f8715640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f00d01e0 con 0x7f00f015c760 2024-03-29T04:07:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.017+0000 7f00f8715640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f00f00b6050 con 0x7f00f006c520 2024-03-29T04:07:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.017+0000 7f00f8715640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f00f006ceb0 con 0x7f00f0072dc0 2024-03-29T04:07:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.018+0000 7f00f5cb0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00f0072dc0 0x7f00f00cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.018+0000 7f00f6cb2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f00f006c520 0x7f00f006c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.018+0000 7f00f64b1640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f00f015c760 0x7f00f006f9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:53974/0 (socket says 172.21.15.97:53974) 2024-03-29T04:07:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.018+0000 7f00f6cb2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f00f006c520 0x7f00f006c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:60734/0 (socket says 172.21.15.97:60734) 2024-03-29T04:07:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.018+0000 7f00f64b1640 1 -- 172.21.15.97:0/677543830 learned_addr learned my addr 172.21.15.97:0/677543830 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.018+0000 7f00f5cb0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00f0072dc0 0x7f00f00cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:48768/0 (socket says 172.21.15.97:48768) 2024-03-29T04:07:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.018+0000 7f00f5cb0640 1 -- 172.21.15.97:0/677543830 >> v2:172.21.15.154:3300/0 conn(0x7f00f006c520 msgr2=0x7f00f006c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.018+0000 7f00f5cb0640 1 --2- 172.21.15.97:0/677543830 >> v2:172.21.15.154:3300/0 conn(0x7f00f006c520 0x7f00f006c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.018+0000 7f00f5cb0640 1 -- 172.21.15.97:0/677543830 >> v1:172.21.15.154:6789/0 conn(0x7f00f015c760 legacy=0x7f00f006f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:51.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.018+0000 7f00f5cb0640 1 -- 172.21.15.97:0/677543830 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f00d04c0 con 0x7f00f0072dc0 2024-03-29T04:07:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.019+0000 7f00f5cb0640 1 --2- 172.21.15.97:0/677543830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00f0072dc0 0x7f00f00cf970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f00e00097a0 tx=0x7f00e002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bac6ed85c951b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.019+0000 7f00f54af640 1 -- 172.21.15.97:0/677543830 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0035020 con 0x7f00f0072dc0 2024-03-29T04:07:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.019+0000 7f00f54af640 1 -- 172.21.15.97:0/677543830 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e0031830 con 0x7f00f0072dc0 2024-03-29T04:07:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.019+0000 7f00f54af640 1 -- 172.21.15.97:0/677543830 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0031c00 con 0x7f00f0072dc0 2024-03-29T04:07:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.019+0000 7f00f8715640 1 -- 172.21.15.97:0/677543830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00f0072dc0 msgr2=0x7f00f00cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.019+0000 7f00f8715640 1 --2- 172.21.15.97:0/677543830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00f0072dc0 0x7f00f00cf970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f00e00097a0 tx=0x7f00e002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.019+0000 7f00f8715640 1 -- 172.21.15.97:0/677543830 shutdown_connections 2024-03-29T04:07:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.019+0000 7f00f8715640 1 --2- 172.21.15.97:0/677543830 >> v2:172.21.15.154:3300/0 conn(0x7f00f006c520 0x7f00f006c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.019+0000 7f00f8715640 1 --2- 172.21.15.97:0/677543830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00f0072dc0 0x7f00f00cf970 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.019+0000 7f00f8715640 1 -- 172.21.15.97:0/677543830 >> 172.21.15.97:0/677543830 conn(0x7f00f0072090 msgr2=0x7f00f00724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.019+0000 7f00f8715640 1 -- 172.21.15.97:0/677543830 shutdown_connections 2024-03-29T04:07:52.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.019+0000 7f00f8715640 1 -- 172.21.15.97:0/677543830 wait complete. 2024-03-29T04:07:52.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.021+0000 7f00f8715640 1 Processor -- start 2024-03-29T04:07:52.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.021+0000 7f00f8715640 1 -- start start 2024-03-29T04:07:52.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.021+0000 7f00f8715640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f006c520 0x7f00f0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.021+0000 7f00f8715640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00f0072dc0 0x7f00f015a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.021+0000 7f00f8715640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00f015c760 0x7f00f0145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.021+0000 7f00f8715640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f00f00cfeb0 con 0x7f00f0072dc0 2024-03-29T04:07:52.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.021+0000 7f00f8715640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f00f006f480 con 0x7f00f006c520 2024-03-29T04:07:52.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.021+0000 7f00f8715640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f00f00d07a0 con 0x7f00f015c760 2024-03-29T04:07:52.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.021+0000 7f00f5cb0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00f0072dc0 0x7f00f015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.021+0000 7f00f5cb0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00f0072dc0 0x7f00f015a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52102/0 (socket says 172.21.15.97:52102) 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.021+0000 7f00f5cb0640 1 -- 172.21.15.97:0/1701276082 learned_addr learned my addr 172.21.15.97:0/1701276082 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.021+0000 7f00f6cb2640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00f015c760 0x7f00f0145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.021+0000 7f00f64b1640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f006c520 0x7f00f0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.022+0000 7f00f5cb0640 1 -- 172.21.15.97:0/1701276082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00f015c760 msgr2=0x7f00f0145370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.022+0000 7f00f5cb0640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00f015c760 0x7f00f0145370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.022+0000 7f00f5cb0640 1 -- 172.21.15.97:0/1701276082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f006c520 msgr2=0x7f00f0156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.022+0000 7f00f5cb0640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f006c520 0x7f00f0156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.022+0000 7f00f5cb0640 1 -- 172.21.15.97:0/1701276082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f0155b20 con 0x7f00f0072dc0 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.022+0000 7f00f6cb2640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00f015c760 0x7f00f0145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.022+0000 7f00f5cb0640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00f0072dc0 0x7f00f015a680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f00e002bee0 tx=0x7f00e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.022+0000 7f00e6ffd640 1 -- 172.21.15.97:0/1701276082 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0035040 con 0x7f00f0072dc0 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.022+0000 7f00e6ffd640 1 -- 172.21.15.97:0/1701276082 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e003b920 con 0x7f00f0072dc0 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.022+0000 7f00e6ffd640 1 -- 172.21.15.97:0/1701276082 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e003bd10 con 0x7f00f0072dc0 2024-03-29T04:07:52.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.022+0000 7f00f8715640 1 -- 172.21.15.97:0/1701276082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f0155e00 con 0x7f00f0072dc0 2024-03-29T04:07:52.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.022+0000 7f00f8715640 1 -- 172.21.15.97:0/1701276082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f01563c0 con 0x7f00f0072dc0 2024-03-29T04:07:52.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.023+0000 7f00f8715640 1 -- 172.21.15.97:0/1701276082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f004f9d0 con 0x7f00f0072dc0 2024-03-29T04:07:52.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.023+0000 7f00e6ffd640 1 -- 172.21.15.97:0/1701276082 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f00e0038020 con 0x7f00f0072dc0 2024-03-29T04:07:52.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.024+0000 7f00e6ffd640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f00d406c560 0x7f00d406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.024+0000 7f00f64b1640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f00d406c560 0x7f00d406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.024+0000 7f00e6ffd640 1 -- 172.21.15.97:0/1701276082 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f00e00ad670 con 0x7f00f0072dc0 2024-03-29T04:07:52.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.024+0000 7f00f64b1640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f00d406c560 0x7f00d406ea20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f00ec007fd0 tx=0x7f00ec007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:52.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.028+0000 7f00e6ffd640 1 -- 172.21.15.97:0/1701276082 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f00e0031800 con 0x7f00f0072dc0 2024-03-29T04:07:52.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.201+0000 7f00f8715640 1 -- 172.21.15.97:0/1701276082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f00f00b2250 con 0x7f00f0072dc0 2024-03-29T04:07:52.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.202+0000 7f00e6ffd640 1 -- 172.21.15.97:0/1701276082 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f00e007b650 con 0x7f00f0072dc0 2024-03-29T04:07:52.182 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:07:52.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.204+0000 7f00f8715640 1 -- 172.21.15.97:0/1701276082 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f00d406c560 msgr2=0x7f00d406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.204+0000 7f00f8715640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f00d406c560 0x7f00d406ea20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f00ec007fd0 tx=0x7f00ec007e80 comp rx=0 tx=0).stop 2024-03-29T04:07:52.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.204+0000 7f00f8715640 1 -- 172.21.15.97:0/1701276082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00f0072dc0 msgr2=0x7f00f015a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.204+0000 7f00f8715640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00f0072dc0 0x7f00f015a680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f00e002bee0 tx=0x7f00e00046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.204+0000 7f00f8715640 1 -- 172.21.15.97:0/1701276082 shutdown_connections 2024-03-29T04:07:52.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.204+0000 7f00f8715640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f00d406c560 0x7f00d406ea20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.204+0000 7f00f8715640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00f015c760 0x7f00f0145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.204+0000 7f00f8715640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00f0072dc0 0x7f00f015a680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.204+0000 7f00f8715640 1 --2- 172.21.15.97:0/1701276082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f006c520 0x7f00f0156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.204+0000 7f00f8715640 1 -- 172.21.15.97:0/1701276082 >> 172.21.15.97:0/1701276082 conn(0x7f00f0072090 msgr2=0x7f00f00b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:52.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.204+0000 7f00f8715640 1 -- 172.21.15.97:0/1701276082 shutdown_connections 2024-03-29T04:07:52.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.204+0000 7f00f8715640 1 -- 172.21.15.97:0/1701276082 wait complete. 2024-03-29T04:07:52.194 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.314+0000 7fe38ef00640 1 Processor -- start 2024-03-29T04:07:52.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.314+0000 7fe38ef00640 1 -- start start 2024-03-29T04:07:52.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe38ef00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe38815c790 0x7fe3880d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe38ef00640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3880ce120 0x7fe3880d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe38ef00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe38815cb70 con 0x7fe38809aa30 2024-03-29T04:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe38ef00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe38809ba70 con 0x7fe38815c790 2024-03-29T04:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe38ef00640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe3880cdc90 con 0x7fe3880ce120 2024-03-29T04:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe38cc9c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe38815c790 0x7fe3880d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe38d49d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe38809aa30 0x7fe388160280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35660/0 (socket says 172.21.15.97:35660) 2024-03-29T04:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe38d49d640 1 -- 172.21.15.97:0/2764239921 learned_addr learned my addr 172.21.15.97:0/2764239921 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe387fff640 1 --2- 172.21.15.97:0/2764239921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3880ce120 0x7fe3880d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe38cc9c640 1 -- 172.21.15.97:0/2764239921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3880ce120 msgr2=0x7fe3880d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe38cc9c640 1 --2- 172.21.15.97:0/2764239921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3880ce120 0x7fe3880d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe38cc9c640 1 -- 172.21.15.97:0/2764239921 >> v1:172.21.15.97:6789/0 conn(0x7fe38809aa30 legacy=0x7fe388160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe38cc9c640 1 -- 172.21.15.97:0/2764239921 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3880cd080 con 0x7fe38815c790 2024-03-29T04:07:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.315+0000 7fe3877fe640 1 -- 172.21.15.97:0/2764239921 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 277702028 0 0) 0x7fe38815cb70 con 0x7fe38809aa30 2024-03-29T04:07:52.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.316+0000 7fe38cc9c640 1 --2- 172.21.15.97:0/2764239921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38815c790 0x7fe3880d09f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0097a0 tx=0x7fe37c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e1b19af9200401 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:52.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.316+0000 7fe3877fe640 1 -- 172.21.15.97:0/2764239921 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c035020 con 0x7fe38815c790 2024-03-29T04:07:52.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.316+0000 7fe3877fe640 1 -- 172.21.15.97:0/2764239921 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe37c031830 con 0x7fe38815c790 2024-03-29T04:07:52.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.316+0000 7fe3877fe640 1 -- 172.21.15.97:0/2764239921 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c031c00 con 0x7fe38815c790 2024-03-29T04:07:52.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.316+0000 7fe38ef00640 1 -- 172.21.15.97:0/2764239921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38815c790 msgr2=0x7fe3880d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.316+0000 7fe38ef00640 1 --2- 172.21.15.97:0/2764239921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38815c790 0x7fe3880d09f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0097a0 tx=0x7fe37c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.316+0000 7fe38ef00640 1 -- 172.21.15.97:0/2764239921 shutdown_connections 2024-03-29T04:07:52.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.316+0000 7fe38ef00640 1 --2- 172.21.15.97:0/2764239921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3880ce120 0x7fe3880d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.316+0000 7fe38ef00640 1 --2- 172.21.15.97:0/2764239921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38815c790 0x7fe3880d09f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.316+0000 7fe38ef00640 1 -- 172.21.15.97:0/2764239921 >> 172.21.15.97:0/2764239921 conn(0x7fe3880b3f00 msgr2=0x7fe3880b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:52.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.316+0000 7fe38ef00640 1 -- 172.21.15.97:0/2764239921 shutdown_connections 2024-03-29T04:07:52.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.316+0000 7fe38ef00640 1 -- 172.21.15.97:0/2764239921 wait complete. 2024-03-29T04:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.318+0000 7fe38ef00640 1 Processor -- start 2024-03-29T04:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.318+0000 7fe38ef00640 1 -- start start 2024-03-29T04:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.318+0000 7fe38ef00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38809aa30 0x7fe388148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.318+0000 7fe38ef00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3880ce120 0x7fe3880b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.318+0000 7fe38ef00640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe38815c790 0x7fe388070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.318+0000 7fe38ef00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe3880cd920 con 0x7fe38809aa30 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.318+0000 7fe38ef00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe3880cd640 con 0x7fe3880ce120 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.318+0000 7fe38ef00640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe38809aee0 con 0x7fe38815c790 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.318+0000 7fe38cc9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38809aa30 0x7fe388148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe38cc9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38809aa30 0x7fe388148960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52120/0 (socket says 172.21.15.97:52120) 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe38cc9c640 1 -- 172.21.15.97:0/1235930468 learned_addr learned my addr 172.21.15.97:0/1235930468 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe387fff640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3880ce120 0x7fe3880b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe38d49d640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe38815c790 0x7fe388070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe38cc9c640 1 -- 172.21.15.97:0/1235930468 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe38815c790 msgr2=0x7fe388070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe38cc9c640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe38815c790 0x7fe388070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe38cc9c640 1 -- 172.21.15.97:0/1235930468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3880ce120 msgr2=0x7fe3880b71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe38cc9c640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3880ce120 0x7fe3880b71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe38cc9c640 1 -- 172.21.15.97:0/1235930468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe388148ea0 con 0x7fe38809aa30 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe387fff640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3880ce120 0x7fe3880b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:52.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe38cc9c640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38809aa30 0x7fe388148960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0098c0 tx=0x7fe37c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:52.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe3857fa640 1 -- 172.21.15.97:0/1235930468 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c03a970 con 0x7fe38809aa30 2024-03-29T04:07:52.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe3857fa640 1 -- 172.21.15.97:0/1235930468 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe37c03ab50 con 0x7fe38809aa30 2024-03-29T04:07:52.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe38ef00640 1 -- 172.21.15.97:0/1235930468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3881f2280 con 0x7fe38809aa30 2024-03-29T04:07:52.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe3857fa640 1 -- 172.21.15.97:0/1235930468 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c045450 con 0x7fe38809aa30 2024-03-29T04:07:52.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.319+0000 7fe38ef00640 1 -- 172.21.15.97:0/1235930468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3881460a0 con 0x7fe38809aa30 2024-03-29T04:07:52.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.320+0000 7fe38ef00640 1 -- 172.21.15.97:0/1235930468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe388068d50 con 0x7fe38809aa30 2024-03-29T04:07:52.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.320+0000 7fe3857fa640 1 -- 172.21.15.97:0/1235930468 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe37c038020 con 0x7fe38809aa30 2024-03-29T04:07:52.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.321+0000 7fe3857fa640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe36c06c530 0x7fe36c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.321+0000 7fe387fff640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe36c06c530 0x7fe36c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.321+0000 7fe3857fa640 1 -- 172.21.15.97:0/1235930468 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe37c0ad3e0 con 0x7fe38809aa30 2024-03-29T04:07:52.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.321+0000 7fe387fff640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe36c06c530 0x7fe36c06e9f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe3881f58d0 tx=0x7fe374009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.325+0000 7fe3857fa640 1 -- 172.21.15.97:0/1235930468 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe37c031800 con 0x7fe38809aa30 2024-03-29T04:07:52.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.540+0000 7fe38ef00640 1 -- 172.21.15.97:0/1235930468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3880b2fc0 con 0x7fe38809aa30 2024-03-29T04:07:52.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.541+0000 7fe3857fa640 1 -- 172.21.15.97:0/1235930468 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7fe37c07b3b0 con 0x7fe38809aa30 2024-03-29T04:07:52.521 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:52.521 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1127043830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127043830},{"type":"v1","addr":"172.21.15.154:6834","nonce":1127043830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3767188750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3767188750},{"type":"v1","addr":"172.21.15.97:6835","nonce":3767188750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3651340478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3651340478},{"type":"v1","addr":"172.21.15.97:6837","nonce":3651340478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:07:40.761466+0000","modified":"2024-03-29T04:07:42.771721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3669418265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3669418265},{"type":"v1","addr":"172.21.15.154:6835","nonce":3669418265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-29T04:07:52.521 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-03-29T04:07:52.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.543+0000 7fe38ef00640 1 -- 172.21.15.97:0/1235930468 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe36c06c530 msgr2=0x7fe36c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.543+0000 7fe38ef00640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe36c06c530 0x7fe36c06e9f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe3881f58d0 tx=0x7fe374009190 comp rx=0 tx=0).stop 2024-03-29T04:07:52.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.543+0000 7fe38ef00640 1 -- 172.21.15.97:0/1235930468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38809aa30 msgr2=0x7fe388148960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.543+0000 7fe38ef00640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38809aa30 0x7fe388148960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0098c0 tx=0x7fe37c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.543+0000 7fe38ef00640 1 -- 172.21.15.97:0/1235930468 shutdown_connections 2024-03-29T04:07:52.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.543+0000 7fe38ef00640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe36c06c530 0x7fe36c06e9f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.543+0000 7fe38ef00640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe38815c790 0x7fe388070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.543+0000 7fe38ef00640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3880ce120 0x7fe3880b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.543+0000 7fe38ef00640 1 --2- 172.21.15.97:0/1235930468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38809aa30 0x7fe388148960 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.543+0000 7fe38ef00640 1 -- 172.21.15.97:0/1235930468 >> 172.21.15.97:0/1235930468 conn(0x7fe3880b3f00 msgr2=0x7fe3880d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:52.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.543+0000 7fe38ef00640 1 -- 172.21.15.97:0/1235930468 shutdown_connections 2024-03-29T04:07:52.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.543+0000 7fe38ef00640 1 -- 172.21.15.97:0/1235930468 wait complete. 2024-03-29T04:07:52.534 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:07:52.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.655+0000 7f76a86b8640 1 Processor -- start 2024-03-29T04:07:52.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.655+0000 7f76a86b8640 1 -- start start 2024-03-29T04:07:52.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a86b8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76a015c670 0x7f76a006c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a86b8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f76a00ad940 0x7f76a006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a86b8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a0146230 con 0x7f76a006f060 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a86b8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f76a006dfc0 con 0x7f76a00ad940 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a86b8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f76a006e2a0 con 0x7f76a015c670 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a5c53640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f76a006f060 0x7f76a00ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35670/0 (socket says 172.21.15.97:35670) 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a5c53640 1 -- 172.21.15.97:0/1159581886 learned_addr learned my addr 172.21.15.97:0/1159581886 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a6c55640 1 --2- 172.21.15.97:0/1159581886 >> v2:172.21.15.154:3300/0 conn(0x7f76a00ad940 0x7f76a006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a6454640 1 --2- 172.21.15.97:0/1159581886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76a015c670 0x7f76a006c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a5452640 1 -- 172.21.15.97:0/1159581886 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237659269 0 0) 0x7f7690001110 con 0x7f76a006f060 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a5452640 1 -- 172.21.15.97:0/1159581886 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f768c003520 con 0x7f76a006f060 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a6454640 1 -- 172.21.15.97:0/1159581886 >> v2:172.21.15.154:3300/0 conn(0x7f76a00ad940 msgr2=0x7f76a006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a6454640 1 --2- 172.21.15.97:0/1159581886 >> v2:172.21.15.154:3300/0 conn(0x7f76a00ad940 0x7f76a006d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a6454640 1 -- 172.21.15.97:0/1159581886 >> v1:172.21.15.97:6789/0 conn(0x7f76a006f060 legacy=0x7f76a00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a6454640 1 -- 172.21.15.97:0/1159581886 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a006e580 con 0x7f76a015c670 2024-03-29T04:07:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.656+0000 7f76a6c55640 1 --2- 172.21.15.97:0/1159581886 >> v2:172.21.15.154:3300/0 conn(0x7f76a00ad940 0x7f76a006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.657+0000 7f76a6454640 1 --2- 172.21.15.97:0/1159581886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76a015c670 0x7f76a006c030 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f769c009d90 tx=0x7f769c02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2900cec7443d847 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.657+0000 7f76a5452640 1 -- 172.21.15.97:0/1159581886 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c039070 con 0x7f76a015c670 2024-03-29T04:07:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.657+0000 7f76a5452640 1 -- 172.21.15.97:0/1159581886 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f769c02cbc0 con 0x7f76a015c670 2024-03-29T04:07:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.657+0000 7f76a5452640 1 -- 172.21.15.97:0/1159581886 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c034440 con 0x7f76a015c670 2024-03-29T04:07:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.658+0000 7f76a86b8640 1 -- 172.21.15.97:0/1159581886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76a015c670 msgr2=0x7f76a006c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.658+0000 7f76a86b8640 1 --2- 172.21.15.97:0/1159581886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76a015c670 0x7f76a006c030 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f769c009d90 tx=0x7f769c02be10 comp rx=0 tx=0).stop 2024-03-29T04:07:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.658+0000 7f76a86b8640 1 -- 172.21.15.97:0/1159581886 shutdown_connections 2024-03-29T04:07:52.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.658+0000 7f76a86b8640 1 --2- 172.21.15.97:0/1159581886 >> v2:172.21.15.154:3300/0 conn(0x7f76a00ad940 0x7f76a006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.658+0000 7f76a86b8640 1 --2- 172.21.15.97:0/1159581886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76a015c670 0x7f76a006c030 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.658+0000 7f76a86b8640 1 -- 172.21.15.97:0/1159581886 >> 172.21.15.97:0/1159581886 conn(0x7f76a006b510 msgr2=0x7f76a006b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:52.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.658+0000 7f76a86b8640 1 -- 172.21.15.97:0/1159581886 shutdown_connections 2024-03-29T04:07:52.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.658+0000 7f76a86b8640 1 -- 172.21.15.97:0/1159581886 wait complete. 2024-03-29T04:07:52.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.659+0000 7f76a86b8640 1 Processor -- start 2024-03-29T04:07:52.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.659+0000 7f76a86b8640 1 -- start start 2024-03-29T04:07:52.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.660+0000 7f76a86b8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76a006f060 0x7f76a01f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.660+0000 7f76a86b8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76a00ad940 0x7f76a01454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.660+0000 7f76a86b8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a015c670 0x7f76a0099700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.660+0000 7f76a86b8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76a00b32c0 con 0x7f76a015c670 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.660+0000 7f76a86b8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f76a015ca50 con 0x7f76a006f060 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.660+0000 7f76a86b8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f76a00b3440 con 0x7f76a00ad940 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.660+0000 7f76a6c55640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a015c670 0x7f76a0099700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.660+0000 7f76a6c55640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a015c670 0x7f76a0099700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52130/0 (socket says 172.21.15.97:52130) 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.660+0000 7f76a6c55640 1 -- 172.21.15.97:0/3923343034 learned_addr learned my addr 172.21.15.97:0/3923343034 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.660+0000 7f76a5c53640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76a00ad940 0x7f76a01454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.660+0000 7f76a6454640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76a006f060 0x7f76a01f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.661+0000 7f76a6c55640 1 -- 172.21.15.97:0/3923343034 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76a00ad940 msgr2=0x7f76a01454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.661+0000 7f76a6c55640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76a00ad940 0x7f76a01454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.661+0000 7f76a6c55640 1 -- 172.21.15.97:0/3923343034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76a006f060 msgr2=0x7f76a01f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.661+0000 7f76a6c55640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76a006f060 0x7f76a01f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.661+0000 7f76a6c55640 1 -- 172.21.15.97:0/3923343034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a0144440 con 0x7f76a015c670 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.661+0000 7f76a6c55640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a015c670 0x7f76a0099700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f769800e8c0 tx=0x7f769800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.661+0000 7f7696ffd640 1 -- 172.21.15.97:0/3923343034 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698017070 con 0x7f76a015c670 2024-03-29T04:07:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.661+0000 7f7696ffd640 1 -- 172.21.15.97:0/3923343034 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7698004020 con 0x7f76a015c670 2024-03-29T04:07:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.661+0000 7f76a86b8640 1 -- 172.21.15.97:0/3923343034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a0144720 con 0x7f76a015c670 2024-03-29T04:07:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.661+0000 7f7696ffd640 1 -- 172.21.15.97:0/3923343034 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76980043f0 con 0x7f76a015c670 2024-03-29T04:07:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.661+0000 7f76a86b8640 1 -- 172.21.15.97:0/3923343034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a0144ce0 con 0x7f76a015c670 2024-03-29T04:07:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.662+0000 7f76a86b8640 1 -- 172.21.15.97:0/3923343034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a0068640 con 0x7f76a015c670 2024-03-29T04:07:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.662+0000 7f7696ffd640 1 -- 172.21.15.97:0/3923343034 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f769800c940 con 0x7f76a015c670 2024-03-29T04:07:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.663+0000 7f7696ffd640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f768406c530 0x7f768406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.663+0000 7f76a6454640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f768406c530 0x7f768406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.663+0000 7f7696ffd640 1 -- 172.21.15.97:0/3923343034 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f769808b7f0 con 0x7f76a015c670 2024-03-29T04:07:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.663+0000 7f76a6454640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f768406c530 0x7f768406e9f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f769c036000 tx=0x7f769c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:52.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.667+0000 7f7696ffd640 1 -- 172.21.15.97:0/3923343034 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f76980597c0 con 0x7f76a015c670 2024-03-29T04:07:52.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.842+0000 7f76a86b8640 1 -- 172.21.15.97:0/3923343034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76a00b4e60 con 0x7f76a015c670 2024-03-29T04:07:52.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.843+0000 7f7696ffd640 1 -- 172.21.15.97:0/3923343034 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f769805dac0 con 0x7f76a015c670 2024-03-29T04:07:52.823 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:52.824 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:07:52.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.845+0000 7f76a86b8640 1 -- 172.21.15.97:0/3923343034 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f768406c530 msgr2=0x7f768406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.845+0000 7f76a86b8640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f768406c530 0x7f768406e9f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f769c036000 tx=0x7f769c030000 comp rx=0 tx=0).stop 2024-03-29T04:07:52.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.845+0000 7f76a86b8640 1 -- 172.21.15.97:0/3923343034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a015c670 msgr2=0x7f76a0099700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.845+0000 7f76a86b8640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a015c670 0x7f76a0099700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f769800e8c0 tx=0x7f769800ed80 comp rx=0 tx=0).stop 2024-03-29T04:07:52.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.845+0000 7f76a86b8640 1 -- 172.21.15.97:0/3923343034 shutdown_connections 2024-03-29T04:07:52.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.845+0000 7f76a86b8640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f768406c530 0x7f768406e9f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.845+0000 7f76a86b8640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76a015c670 0x7f76a0099700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.845+0000 7f76a86b8640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76a00ad940 0x7f76a01454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.845+0000 7f76a86b8640 1 --2- 172.21.15.97:0/3923343034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76a006f060 0x7f76a01f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.845+0000 7f76a86b8640 1 -- 172.21.15.97:0/3923343034 >> 172.21.15.97:0/3923343034 conn(0x7f76a006b510 msgr2=0x7f76a015a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:52.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.845+0000 7f76a86b8640 1 -- 172.21.15.97:0/3923343034 shutdown_connections 2024-03-29T04:07:52.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.846+0000 7f76a86b8640 1 -- 172.21.15.97:0/3923343034 wait complete. 2024-03-29T04:07:52.836 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:07:52.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.957+0000 7f21ee44e640 1 Processor -- start 2024-03-29T04:07:52.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.957+0000 7f21ee44e640 1 -- start start 2024-03-29T04:07:52.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21ee44e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21e815e9a0 0x7f21e8072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21ee44e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f21e806c520 0x7f21e806c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21ee44e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21e8073240 con 0x7f21e806e0f0 2024-03-29T04:07:52.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21ee44e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f21e806cec0 con 0x7f21e806c520 2024-03-29T04:07:52.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21ee44e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f21e806f2c0 con 0x7f21e815e9a0 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21ec9eb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f21e806c520 0x7f21e806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21e77fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f21e806e0f0 0x7f21e806f750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:53988/0 (socket says 172.21.15.97:53988) 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21e77fe640 1 -- 172.21.15.97:0/2983009536 learned_addr learned my addr 172.21.15.97:0/2983009536 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21e7fff640 1 --2- 172.21.15.97:0/2983009536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21e815e9a0 0x7f21e8072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21ec9eb640 1 -- 172.21.15.97:0/2983009536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21e815e9a0 msgr2=0x7f21e8072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21ec9eb640 1 --2- 172.21.15.97:0/2983009536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21e815e9a0 0x7f21e8072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21ec9eb640 1 -- 172.21.15.97:0/2983009536 >> v1:172.21.15.154:6789/0 conn(0x7f21e806e0f0 legacy=0x7f21e806f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21e6ffd640 1 -- 172.21.15.97:0/2983009536 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 675473231 0 0) 0x7f21e8073240 con 0x7f21e806e0f0 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.958+0000 7f21ec9eb640 1 -- 172.21.15.97:0/2983009536 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e806e6b0 con 0x7f21e806c520 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.959+0000 7f21ec9eb640 1 --2- 172.21.15.97:0/2983009536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21e806c520 0x7f21e806c980 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009d90 tx=0x7f21e002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d0c0656ccd9ef2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.959+0000 7f21e6ffd640 1 -- 172.21.15.97:0/2983009536 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0039070 con 0x7f21e806c520 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.959+0000 7f21e6ffd640 1 -- 172.21.15.97:0/2983009536 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e002cbc0 con 0x7f21e806c520 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.959+0000 7f21e6ffd640 1 -- 172.21.15.97:0/2983009536 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0034440 con 0x7f21e806c520 2024-03-29T04:07:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.959+0000 7f21ee44e640 1 -- 172.21.15.97:0/2983009536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21e806c520 msgr2=0x7f21e806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.959+0000 7f21ee44e640 1 --2- 172.21.15.97:0/2983009536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21e806c520 0x7f21e806c980 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009d90 tx=0x7f21e002be10 comp rx=0 tx=0).stop 2024-03-29T04:07:52.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.960+0000 7f21ee44e640 1 -- 172.21.15.97:0/2983009536 shutdown_connections 2024-03-29T04:07:52.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.960+0000 7f21ee44e640 1 --2- 172.21.15.97:0/2983009536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21e806c520 0x7f21e806c980 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.960+0000 7f21ee44e640 1 --2- 172.21.15.97:0/2983009536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21e815e9a0 0x7f21e8072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.960+0000 7f21ee44e640 1 -- 172.21.15.97:0/2983009536 >> 172.21.15.97:0/2983009536 conn(0x7f21e8072090 msgr2=0x7f21e80724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:52.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.960+0000 7f21ee44e640 1 -- 172.21.15.97:0/2983009536 shutdown_connections 2024-03-29T04:07:52.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.960+0000 7f21ee44e640 1 -- 172.21.15.97:0/2983009536 wait complete. 2024-03-29T04:07:52.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.961+0000 7f21ee44e640 1 Processor -- start 2024-03-29T04:07:52.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.961+0000 7f21ee44e640 1 -- start start 2024-03-29T04:07:52.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21ee44e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21e806c520 0x7f21e814cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21ee44e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21e806e0f0 0x7f21e81ea680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21ee44e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21e815e9a0 0x7f21e80afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21ee44e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f21e806ef50 con 0x7f21e806e0f0 2024-03-29T04:07:52.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21ee44e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f21e806ec70 con 0x7f21e806c520 2024-03-29T04:07:52.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21ee44e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f21e806fe60 con 0x7f21e815e9a0 2024-03-29T04:07:52.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21e806c520 0x7f21e814cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21e806e0f0 0x7f21e81ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21e806c520 0x7f21e814cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:60792/0 (socket says 172.21.15.97:60792) 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e7fff640 1 -- 172.21.15.97:0/2632936800 learned_addr learned my addr 172.21.15.97:0/2632936800 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21ec9eb640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21e815e9a0 0x7f21e80afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e7fff640 1 -- 172.21.15.97:0/2632936800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21e815e9a0 msgr2=0x7f21e80afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e7fff640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21e815e9a0 0x7f21e80afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e7fff640 1 -- 172.21.15.97:0/2632936800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21e806e0f0 msgr2=0x7f21e81ea680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e7fff640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21e806e0f0 0x7f21e81ea680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e7fff640 1 -- 172.21.15.97:0/2632936800 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e814d010 con 0x7f21e806c520 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e77fe640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21e806e0f0 0x7f21e81ea680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e7fff640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21e806c520 0x7f21e814cad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00e870 tx=0x7f21dc00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e4ff9640 1 -- 172.21.15.97:0/2632936800 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc017070 con 0x7f21e806c520 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e4ff9640 1 -- 172.21.15.97:0/2632936800 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21dc00c940 con 0x7f21e806c520 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21ee44e640 1 -- 172.21.15.97:0/2632936800 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e814a530 con 0x7f21e806c520 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21e4ff9640 1 -- 172.21.15.97:0/2632936800 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc00cd30 con 0x7f21e806c520 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.962+0000 7f21ee44e640 1 -- 172.21.15.97:0/2632936800 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e814aaf0 con 0x7f21e806c520 2024-03-29T04:07:52.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.963+0000 7f21ee44e640 1 -- 172.21.15.97:0/2632936800 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e8068640 con 0x7f21e806c520 2024-03-29T04:07:52.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.964+0000 7f21e4ff9640 1 -- 172.21.15.97:0/2632936800 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f21dc01a070 con 0x7f21e806c520 2024-03-29T04:07:52.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.964+0000 7f21e4ff9640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f21d406c560 0x7f21d406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:52.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.964+0000 7f21e77fe640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f21d406c560 0x7f21d406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:52.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.964+0000 7f21e4ff9640 1 -- 172.21.15.97:0/2632936800 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f21dc08b4b0 con 0x7f21e806c520 2024-03-29T04:07:52.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.965+0000 7f21e77fe640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f21d406c560 0x7f21d406ea20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f21e80c7300 tx=0x7f21d8007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:52.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:52.969+0000 7f21e4ff9640 1 -- 172.21.15.97:0/2632936800 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f21dc059480 con 0x7f21e806c520 2024-03-29T04:07:53.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.179+0000 7f21ee44e640 1 -- 172.21.15.97:0/2632936800 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21e80b3fb0 con 0x7f21e806c520 2024-03-29T04:07:53.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.179+0000 7f21e4ff9640 1 -- 172.21.15.97:0/2632936800 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7f21dc05d780 con 0x7f21e806c520 2024-03-29T04:07:53.159 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:07:53.159 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1127043830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127043830},{"type":"v1","addr":"172.21.15.154:6834","nonce":1127043830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3767188750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3767188750},{"type":"v1","addr":"172.21.15.97:6835","nonce":3767188750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3651340478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3651340478},{"type":"v1","addr":"172.21.15.97:6837","nonce":3651340478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:07:40.761466+0000","modified":"2024-03-29T04:07:42.771721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3669418265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3669418265},{"type":"v1","addr":"172.21.15.154:6835","nonce":3669418265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-29T04:07:53.159 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-03-29T04:07:53.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.181+0000 7f21ee44e640 1 -- 172.21.15.97:0/2632936800 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f21d406c560 msgr2=0x7f21d406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:53.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.181+0000 7f21ee44e640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f21d406c560 0x7f21d406ea20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f21e80c7300 tx=0x7f21d8007d50 comp rx=0 tx=0).stop 2024-03-29T04:07:53.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.181+0000 7f21ee44e640 1 -- 172.21.15.97:0/2632936800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21e806c520 msgr2=0x7f21e814cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:53.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.181+0000 7f21ee44e640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21e806c520 0x7f21e814cad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00e870 tx=0x7f21dc00ed30 comp rx=0 tx=0).stop 2024-03-29T04:07:53.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.181+0000 7f21ee44e640 1 -- 172.21.15.97:0/2632936800 shutdown_connections 2024-03-29T04:07:53.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.181+0000 7f21ee44e640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f21d406c560 0x7f21d406ea20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.181+0000 7f21ee44e640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21e815e9a0 0x7f21e80afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.181+0000 7f21ee44e640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21e806e0f0 0x7f21e81ea680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.181+0000 7f21ee44e640 1 --2- 172.21.15.97:0/2632936800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21e806c520 0x7f21e814cad0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.181+0000 7f21ee44e640 1 -- 172.21.15.97:0/2632936800 >> 172.21.15.97:0/2632936800 conn(0x7f21e8072090 msgr2=0x7f21e81620c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:53.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.181+0000 7f21ee44e640 1 -- 172.21.15.97:0/2632936800 shutdown_connections 2024-03-29T04:07:53.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.181+0000 7f21ee44e640 1 -- 172.21.15.97:0/2632936800 wait complete. 2024-03-29T04:07:53.575 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T04:07:40.761466+0000', 'modified': '2024-03-29T04:07:42.771721+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4297}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4297': {'gid': 4297, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.154:6835/3669418265', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6833', 'nonce': 3669418265}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 3669418265}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4297, 'qdb_cluster': [4297]} 2024-03-29T04:07:53.575 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T04:07:53.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-29T04:07:53.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.698+0000 7f9b65359640 1 Processor -- start 2024-03-29T04:07:53.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.698+0000 7f9b65359640 1 -- start start 2024-03-29T04:07:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.699+0000 7f9b65359640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9b600d3030 0x7f9b60169d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.699+0000 7f9b65359640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b600cf310 con 0x7f9b600d2ba0 2024-03-29T04:07:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.699+0000 7f9b65359640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b600d34c0 con 0x7f9b6015e6b0 2024-03-29T04:07:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.699+0000 7f9b65359640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f9b6006e640 con 0x7f9b600d3030 2024-03-29T04:07:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.699+0000 7f9b5e7fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9b600d2ba0 0x7f9b60166650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35680/0 (socket says 172.21.15.97:35680) 2024-03-29T04:07:53.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.699+0000 7f9b5e7fc640 1 -- 172.21.15.97:0/1962164859 learned_addr learned my addr 172.21.15.97:0/1962164859 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.699+0000 7f9b5f7fe640 1 --2- 172.21.15.97:0/1962164859 >> v2:172.21.15.154:3300/0 conn(0x7f9b600d3030 0x7f9b60169d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.699+0000 7f9b5dffb640 1 -- 172.21.15.97:0/1962164859 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 729046356 0 0) 0x7f9b600cf310 con 0x7f9b600d2ba0 2024-03-29T04:07:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.699+0000 7f9b5dffb640 1 -- 172.21.15.97:0/1962164859 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b48003520 con 0x7f9b600d2ba0 2024-03-29T04:07:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.699+0000 7f9b5dffb640 1 -- 172.21.15.97:0/1962164859 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 601062536 0 0) 0x7f9b600d34c0 con 0x7f9b6015e6b0 2024-03-29T04:07:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.699+0000 7f9b5dffb640 1 -- 172.21.15.97:0/1962164859 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b600cf310 con 0x7f9b6015e6b0 2024-03-29T04:07:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.700+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/1962164859 >> v1:172.21.15.154:6789/0 conn(0x7f9b6015e6b0 legacy=0x7f9b600d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.700+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/1962164859 >> v1:172.21.15.97:6789/0 conn(0x7f9b600d2ba0 legacy=0x7f9b60166650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.700+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/1962164859 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b600d4f70 con 0x7f9b600d3030 2024-03-29T04:07:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.700+0000 7f9b5f7fe640 1 --2- 172.21.15.97:0/1962164859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b600d3030 0x7f9b60169d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9b580097d0 tx=0x7f9b5802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e1895da67f45e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.700+0000 7f9b5dffb640 1 -- 172.21.15.97:0/1962164859 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58034070 con 0x7f9b600d3030 2024-03-29T04:07:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.700+0000 7f9b5dffb640 1 -- 172.21.15.97:0/1962164859 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b58030610 con 0x7f9b600d3030 2024-03-29T04:07:53.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.700+0000 7f9b5dffb640 1 -- 172.21.15.97:0/1962164859 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b580309e0 con 0x7f9b600d3030 2024-03-29T04:07:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.701+0000 7f9b65359640 1 -- 172.21.15.97:0/1962164859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b600d3030 msgr2=0x7f9b60169d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.701+0000 7f9b65359640 1 --2- 172.21.15.97:0/1962164859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b600d3030 0x7f9b60169d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9b580097d0 tx=0x7f9b5802b960 comp rx=0 tx=0).stop 2024-03-29T04:07:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.701+0000 7f9b65359640 1 -- 172.21.15.97:0/1962164859 shutdown_connections 2024-03-29T04:07:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.701+0000 7f9b65359640 1 --2- 172.21.15.97:0/1962164859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b600d3030 0x7f9b60169d80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.701+0000 7f9b65359640 1 -- 172.21.15.97:0/1962164859 >> 172.21.15.97:0/1962164859 conn(0x7f9b6008bfc0 msgr2=0x7f9b600d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.701+0000 7f9b65359640 1 -- 172.21.15.97:0/1962164859 shutdown_connections 2024-03-29T04:07:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.701+0000 7f9b65359640 1 -- 172.21.15.97:0/1962164859 wait complete. 2024-03-29T04:07:53.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.702+0000 7f9b65359640 1 Processor -- start 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.702+0000 7f9b65359640 1 -- start start 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b65359640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b600d2ba0 0x7f9b601549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b65359640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9b600d3030 0x7f9b600b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b65359640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b6015e6b0 0x7f9b600ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b65359640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9b600cfb70 con 0x7f9b6015e6b0 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b65359640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9b600cf890 con 0x7f9b600d2ba0 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b65359640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9b600cfe50 con 0x7f9b600d3030 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b6015e6b0 0x7f9b600ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b6015e6b0 0x7f9b600ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52152/0 (socket says 172.21.15.97:52152) 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/4189386293 learned_addr learned my addr 172.21.15.97:0/4189386293 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b5e7fc640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9b600d3030 0x7f9b600b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b5effd640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b600d2ba0 0x7f9b601549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/4189386293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9b600d3030 msgr2=0x7f9b600b71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b5f7fe640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9b600d3030 0x7f9b600b71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/4189386293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b600d2ba0 msgr2=0x7f9b601549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b5f7fe640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b600d2ba0 0x7f9b601549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/4189386293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b60154f30 con 0x7f9b6015e6b0 2024-03-29T04:07:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.703+0000 7f9b5f7fe640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b6015e6b0 0x7f9b600ca930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b580097d0 tx=0x7f9b580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.704+0000 7f9b477fe640 1 -- 172.21.15.97:0/4189386293 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58034040 con 0x7f9b6015e6b0 2024-03-29T04:07:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.704+0000 7f9b477fe640 1 -- 172.21.15.97:0/4189386293 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b58030410 con 0x7f9b6015e6b0 2024-03-29T04:07:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.704+0000 7f9b477fe640 1 -- 172.21.15.97:0/4189386293 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58006e10 con 0x7f9b6015e6b0 2024-03-29T04:07:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.704+0000 7f9b65359640 1 -- 172.21.15.97:0/4189386293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b60159860 con 0x7f9b6015e6b0 2024-03-29T04:07:53.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.704+0000 7f9b65359640 1 -- 172.21.15.97:0/4189386293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b60159e20 con 0x7f9b6015e6b0 2024-03-29T04:07:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.705+0000 7f9b477fe640 1 -- 172.21.15.97:0/4189386293 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9b58037070 con 0x7f9b6015e6b0 2024-03-29T04:07:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.705+0000 7f9b477fe640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9b4c06c560 0x7f9b4c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.706+0000 7f9b477fe640 1 -- 172.21.15.97:0/4189386293 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9b580ad040 con 0x7f9b6015e6b0 2024-03-29T04:07:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.706+0000 7f9b5effd640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9b4c06c560 0x7f9b4c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.706+0000 7f9b5effd640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9b4c06c560 0x7f9b4c06ea20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9b60162260 tx=0x7f9b50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.706+0000 7f9b68027740 1 Processor -- start 2024-03-29T04:07:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.707+0000 7f9b68027740 1 -- start start 2024-03-29T04:07:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.707+0000 7f9b68027740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86860 0x55b97be59800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.707+0000 7f9b68027740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86c40 0x55b97be895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.707+0000 7f9b68027740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be59d40 0x55b97be89b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.707+0000 7f9b68027740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b97bdc3020 con 0x55b97be86c40 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.707+0000 7f9b68027740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b97bd6b5f0 con 0x55b97be59d40 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.707+0000 7f9b68027740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55b97bdbefd0 con 0x55b97be86860 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.707+0000 7f9b5e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86c40 0x55b97be895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.707+0000 7f9b5e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86c40 0x55b97be895f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52160/0 (socket says 172.21.15.97:52160) 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.707+0000 7f9b5e7fc640 1 -- 172.21.15.97:0/2927067739 learned_addr learned my addr 172.21.15.97:0/2927067739 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.707+0000 7f9b5effd640 1 --2- 172.21.15.97:0/2927067739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86860 0x55b97be59800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.707+0000 7f9b5f7fe640 1 --2- 172.21.15.97:0/2927067739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be59d40 0x55b97be89b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.708+0000 7f9b5e7fc640 1 -- 172.21.15.97:0/2927067739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86860 msgr2=0x55b97be59800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.708+0000 7f9b5e7fc640 1 --2- 172.21.15.97:0/2927067739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86860 0x55b97be59800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.708+0000 7f9b5e7fc640 1 -- 172.21.15.97:0/2927067739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be59d40 msgr2=0x55b97be89b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.708+0000 7f9b5e7fc640 1 --2- 172.21.15.97:0/2927067739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be59d40 0x55b97be89b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.708+0000 7f9b5e7fc640 1 -- 172.21.15.97:0/2927067739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b97bad72d0 con 0x55b97be86c40 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.708+0000 7f9b5e7fc640 1 --2- 172.21.15.97:0/2927067739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86c40 0x55b97be895f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b54001260 tx=0x7f9b54007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8d4fd06eb22c61 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:53.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.708+0000 7f9b44ff9640 1 -- 172.21.15.97:0/2927067739 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b540072e0 con 0x55b97be86c40 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.708+0000 7f9b44ff9640 1 -- 172.21.15.97:0/2927067739 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b54018070 con 0x55b97be86c40 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.708+0000 7f9b44ff9640 1 -- 172.21.15.97:0/2927067739 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b540057a0 con 0x55b97be86c40 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.709+0000 7f9b68027740 1 -- 172.21.15.97:0/2927067739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86c40 msgr2=0x55b97be895f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.709+0000 7f9b68027740 1 --2- 172.21.15.97:0/2927067739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86c40 0x55b97be895f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b54001260 tx=0x7f9b54007c00 comp rx=0 tx=0).stop 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.709+0000 7f9b68027740 1 -- 172.21.15.97:0/2927067739 shutdown_connections 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.709+0000 7f9b68027740 1 --2- 172.21.15.97:0/2927067739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be59d40 0x55b97be89b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.709+0000 7f9b68027740 1 --2- 172.21.15.97:0/2927067739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86c40 0x55b97be895f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.709+0000 7f9b68027740 1 --2- 172.21.15.97:0/2927067739 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86860 0x55b97be59800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.709+0000 7f9b68027740 1 -- 172.21.15.97:0/2927067739 >> 172.21.15.97:0/2927067739 conn(0x55b97be5ba70 msgr2=0x55b97bea45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.709+0000 7f9b68027740 1 -- 172.21.15.97:0/2927067739 shutdown_connections 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.709+0000 7f9b68027740 1 -- 172.21.15.97:0/2927067739 wait complete. 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.709+0000 7f9b68027740 1 -- start start 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.709+0000 7f9b68027740 1 Processor -- start 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b68027740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86860 0x55b97beb53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b68027740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86c40 0x55b97bea4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b68027740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 0x55b97be90370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b68027740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b97be5e110 con 0x55b97be8e450 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b68027740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b97bd7e9c0 con 0x55b97be86860 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b68027740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55b97be60950 con 0x55b97be86c40 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b68027740 10 client.0 _ll_register_callbacks cb 0x55b97bd62fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 0x55b97be90370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 0x55b97be90370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52176/0 (socket says 172.21.15.97:52176) 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/4180116789 learned_addr learned my addr 172.21.15.97:0/4180116789 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b5e7fc640 1 --2- 172.21.15.97:0/4180116789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86c40 0x55b97bea4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b5effd640 1 --2- 172.21.15.97:0/4180116789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86860 0x55b97beb53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/4180116789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86c40 msgr2=0x55b97bea4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b5f7fe640 1 --2- 172.21.15.97:0/4180116789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86c40 0x55b97bea4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/4180116789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86860 msgr2=0x55b97beb53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:53.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b5f7fe640 1 --2- 172.21.15.97:0/4180116789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86860 0x55b97beb53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/4180116789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b97bba16b0 con 0x55b97be8e450 2024-03-29T04:07:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.710+0000 7f9b5f7fe640 1 --2- 172.21.15.97:0/4180116789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 0x55b97be90370 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9b58077500 tx=0x7f9b5807b160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e94ca861ab6fe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-29T04:07:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b3dffb640 1 -- 172.21.15.97:0/4180116789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58083740 con 0x55b97be8e450 2024-03-29T04:07:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b3dffb640 1 -- 172.21.15.97:0/4180116789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9b580838e0 con 0x55b97be8e450 2024-03-29T04:07:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b68027740 1 -- 172.21.15.97:0/4180116789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b54018070 con 0x55b97be8e450 2024-03-29T04:07:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b3dffb640 1 -- 172.21.15.97:0/4180116789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f9b580ab260 con 0x55b97be8e450 2024-03-29T04:07:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b68027740 10 client.4620 fetch_fsmap learned FSMap version 6 2024-03-29T04:07:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b68027740 1 -- 172.21.15.97:0/4180116789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55b97bea4f20 con 0x55b97be8e450 2024-03-29T04:07:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b3dffb640 1 -- 172.21.15.97:0/4180116789 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b54018070 con 0x55b97be8e450 2024-03-29T04:07:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b3dffb640 1 -- 172.21.15.97:0/4180116789 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 6) ==== 3494+0+0 (secure 0 0 0) 0x7f9b58041070 con 0x55b97be8e450 2024-03-29T04:07:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b68027740 10 client.4620 fetch_fsmap finished waiting for FSMap version 6 2024-03-29T04:07:53.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b68027740 10 client.4620 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b68027740 20 client.4620 populate_metadata read hostname 'smithi097' 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b68027740 1 --2- 172.21.15.97:0/4180116789 >> [v2:172.21.15.154:6833/3669418265,v1:172.21.15.154:6835/3669418265] conn(0x55b97bebe510 0x55b97bec2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b68027740 4 client.4620 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b68027740 1 -- 172.21.15.97:0/4180116789 --> [v2:172.21.15.154:6833/3669418265,v1:172.21.15.154:6835/3669418265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b97bd974e0 con 0x55b97bebe510 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.711+0000 7f9b5effd640 1 --2- 172.21.15.97:0/4180116789 >> [v2:172.21.15.154:6833/3669418265,v1:172.21.15.154:6835/3669418265] conn(0x55b97bebe510 0x55b97bec2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b3dffb640 10 client.4620 ms_handle_connect on v2:172.21.15.154:6833/3669418265 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b5effd640 1 --2- 172.21.15.97:0/4180116789 >> [v2:172.21.15.154:6833/3669418265,v1:172.21.15.154:6835/3669418265] conn(0x55b97bebe510 0x55b97bec2830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb9da7cde8af37a server_cookie=ab4a8bfaa93062cd in_seq=0 out_seq=0 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b3dffb640 1 -- 172.21.15.97:0/4180116789 <== mds.0 v2:172.21.15.154:6833/3669418265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55b97bd974e0 con 0x55b97bebe510 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b3dffb640 10 client.4620 handle_command_reply: tid=0 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b68027740 1 client.4620 shutdown 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b68027740 1 -- 172.21.15.97:0/4180116789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 msgr2=0x55b97be90370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b68027740 1 --2- 172.21.15.97:0/4180116789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 0x55b97be90370 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9b58077500 tx=0x7f9b5807b160 comp rx=0 tx=0).stop 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b68027740 1 -- 172.21.15.97:0/4180116789 shutdown_connections 2024-03-29T04:07:53.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b68027740 1 --2- 172.21.15.97:0/4180116789 >> [v2:172.21.15.154:6833/3669418265,v1:172.21.15.154:6835/3669418265] conn(0x55b97bebe510 0x55b97bec2830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b68027740 1 --2- 172.21.15.97:0/4180116789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 0x55b97be90370 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b3dffb640 0 client.4620 ms_handle_reset on v2:172.21.15.154:6833/3669418265 2024-03-29T04:07:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b68027740 1 --2- 172.21.15.97:0/4180116789 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86c40 0x55b97bea4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b68027740 1 --2- 172.21.15.97:0/4180116789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86860 0x55b97beb53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b68027740 1 -- 172.21.15.97:0/4180116789 >> 172.21.15.97:0/4180116789 conn(0x55b97be59d40 msgr2=0x55b97bea7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b68027740 1 -- 172.21.15.97:0/4180116789 shutdown_connections 2024-03-29T04:07:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b68027740 1 -- 172.21.15.97:0/4180116789 wait complete. 2024-03-29T04:07:53.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.712+0000 7f9b68027740 20 client.4620 trim_cache size 0 max 16384 2024-03-29T04:07:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.730+0000 7f9b68027740 1 Processor -- start 2024-03-29T04:07:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.731+0000 7f9b68027740 1 -- start start 2024-03-29T04:07:53.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.731+0000 7f9b68027740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86860 0x55b97bec81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.731+0000 7f9b68027740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86c40 0x55b97be89220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.731+0000 7f9b68027740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 0x55b97beb0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.731+0000 7f9b68027740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9b601ffe30 con 0x55b97be8e450 2024-03-29T04:07:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.731+0000 7f9b68027740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9b600c83b0 con 0x55b97be86860 2024-03-29T04:07:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.731+0000 7f9b68027740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9b600c8a00 con 0x55b97be86c40 2024-03-29T04:07:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.731+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 0x55b97beb0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.731+0000 7f9b5f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 0x55b97beb0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52182/0 (socket says 172.21.15.97:52182) 2024-03-29T04:07:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.731+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/3932327946 learned_addr learned my addr 172.21.15.97:0/3932327946 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.731+0000 7f9b5effd640 1 --2- 172.21.15.97:0/3932327946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86860 0x55b97bec81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.731+0000 7f9b5e7fc640 1 --2- 172.21.15.97:0/3932327946 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86c40 0x55b97be89220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/3932327946 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86c40 msgr2=0x55b97be89220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b5f7fe640 1 --2- 172.21.15.97:0/3932327946 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86c40 0x55b97be89220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/3932327946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86860 msgr2=0x55b97bec81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b5f7fe640 1 --2- 172.21.15.97:0/3932327946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86860 0x55b97bec81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b5f7fe640 1 -- 172.21.15.97:0/3932327946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b97be8cc50 con 0x55b97be8e450 2024-03-29T04:07:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b5f7fe640 1 --2- 172.21.15.97:0/3932327946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 0x55b97beb0c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b580836a0 tx=0x7f9b58092000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f7db8e3d007ac server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b3ffff640 1 -- 172.21.15.97:0/3932327946 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5807abe0 con 0x55b97be8e450 2024-03-29T04:07:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b3ffff640 1 -- 172.21.15.97:0/3932327946 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b58030070 con 0x55b97be8e450 2024-03-29T04:07:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b3ffff640 1 -- 172.21.15.97:0/3932327946 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58083940 con 0x55b97be8e450 2024-03-29T04:07:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b68027740 1 -- 172.21.15.97:0/3932327946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 msgr2=0x55b97beb0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b68027740 1 --2- 172.21.15.97:0/3932327946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 0x55b97beb0c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b580836a0 tx=0x7f9b58092000 comp rx=0 tx=0).stop 2024-03-29T04:07:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b68027740 1 -- 172.21.15.97:0/3932327946 shutdown_connections 2024-03-29T04:07:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b68027740 1 --2- 172.21.15.97:0/3932327946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be8e450 0x55b97beb0c30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b68027740 1 --2- 172.21.15.97:0/3932327946 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be86c40 0x55b97be89220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b68027740 1 --2- 172.21.15.97:0/3932327946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86860 0x55b97bec81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b68027740 1 -- 172.21.15.97:0/3932327946 >> 172.21.15.97:0/3932327946 conn(0x55b97be59d40 msgr2=0x55b97be90e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b68027740 1 -- 172.21.15.97:0/3932327946 shutdown_connections 2024-03-29T04:07:53.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.732+0000 7f9b68027740 1 -- 172.21.15.97:0/3932327946 wait complete. 2024-03-29T04:07:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.733+0000 7f9b68027740 1 -- start start 2024-03-29T04:07:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.733+0000 7f9b68027740 1 Processor -- start 2024-03-29T04:07:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b68027740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86860 0x55b97bf175b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b68027740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86c40 0x55b97bea57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b68027740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be8e450 0x55b97bea5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b68027740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b97bebe390 con 0x55b97be86860 2024-03-29T04:07:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b68027740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b97bebdf40 con 0x55b97be86c40 2024-03-29T04:07:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b68027740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55b97bebe950 con 0x55b97be8e450 2024-03-29T04:07:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b68027740 10 client.0 _ll_register_callbacks cb 0x7f9b5807aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:07:53.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b5effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86860 0x55b97bf175b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b5effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86860 0x55b97bf175b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52188/0 (socket says 172.21.15.97:52188) 2024-03-29T04:07:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b5effd640 1 -- 172.21.15.97:0/1270111889 learned_addr learned my addr 172.21.15.97:0/1270111889 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b5e7fc640 1 --2- 172.21.15.97:0/1270111889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86c40 0x55b97bea57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b5f7fe640 1 --2- 172.21.15.97:0/1270111889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be8e450 0x55b97bea5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b5effd640 1 -- 172.21.15.97:0/1270111889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be8e450 msgr2=0x55b97bea5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:53.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b5effd640 1 --2- 172.21.15.97:0/1270111889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be8e450 0x55b97bea5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b5effd640 1 -- 172.21.15.97:0/1270111889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86c40 msgr2=0x55b97bea57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b5effd640 1 --2- 172.21.15.97:0/1270111889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86c40 0x55b97bea57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b5effd640 1 -- 172.21.15.97:0/1270111889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b97be5ee50 con 0x55b97be86860 2024-03-29T04:07:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b5effd640 1 --2- 172.21.15.97:0/1270111889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86860 0x55b97bf175b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9b50002230 tx=0x7f9b50001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca0c43e2ae524ff server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-29T04:07:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b3dffb640 1 -- 172.21.15.97:0/1270111889 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5002ae10 con 0x55b97be86860 2024-03-29T04:07:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b3dffb640 1 -- 172.21.15.97:0/1270111889 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9b500234d0 con 0x55b97be86860 2024-03-29T04:07:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.734+0000 7f9b68027740 1 -- 172.21.15.97:0/1270111889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b58030070 con 0x55b97be86860 2024-03-29T04:07:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b3dffb640 1 -- 172.21.15.97:0/1270111889 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f9b5003a050 con 0x55b97be86860 2024-03-29T04:07:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b68027740 10 client.4626 fetch_fsmap learned FSMap version 6 2024-03-29T04:07:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b68027740 1 -- 172.21.15.97:0/1270111889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55b97beb0cc0 con 0x55b97be86860 2024-03-29T04:07:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b3dffb640 1 -- 172.21.15.97:0/1270111889 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b58030070 con 0x55b97be86860 2024-03-29T04:07:53.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b3dffb640 1 -- 172.21.15.97:0/1270111889 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 6) ==== 3494+0+0 (secure 0 0 0) 0x7f9b5003d020 con 0x55b97be86860 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b68027740 10 client.4626 fetch_fsmap finished waiting for FSMap version 6 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b68027740 10 client.4626 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b68027740 20 client.4626 populate_metadata read hostname 'smithi097' 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b68027740 1 --2- 172.21.15.97:0/1270111889 >> [v2:172.21.15.154:6833/3669418265,v1:172.21.15.154:6835/3669418265] conn(0x55b97be586f0 0x55b97be58ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b68027740 4 client.4626 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b68027740 1 -- 172.21.15.97:0/1270111889 --> [v2:172.21.15.154:6833/3669418265,v1:172.21.15.154:6835/3669418265] -- command(tid 0: {"prefix": "status"}) -- 0x55b97bdd0560 con 0x55b97be586f0 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b5e7fc640 1 --2- 172.21.15.97:0/1270111889 >> [v2:172.21.15.154:6833/3669418265,v1:172.21.15.154:6835/3669418265] conn(0x55b97be586f0 0x55b97be58ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b3dffb640 10 client.4626 ms_handle_connect on v2:172.21.15.154:6833/3669418265 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b5e7fc640 1 --2- 172.21.15.97:0/1270111889 >> [v2:172.21.15.154:6833/3669418265,v1:172.21.15.154:6835/3669418265] conn(0x55b97be586f0 0x55b97be58ad0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f93871f4eca52198 server_cookie=cc9add96642b735e in_seq=0 out_seq=0 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b3dffb640 1 -- 172.21.15.97:0/1270111889 <== mds.0 v2:172.21.15.154:6833/3669418265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55b97bdd0560 con 0x55b97be586f0 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b3dffb640 10 client.4626 handle_command_reply: tid=0 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.735+0000 7f9b68027740 1 client.4626 shutdown 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b68027740 1 -- 172.21.15.97:0/1270111889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86860 msgr2=0x55b97bf175b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:53.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b68027740 1 --2- 172.21.15.97:0/1270111889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86860 0x55b97bf175b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9b50002230 tx=0x7f9b50001250 comp rx=0 tx=0).stop 2024-03-29T04:07:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b68027740 1 -- 172.21.15.97:0/1270111889 shutdown_connections 2024-03-29T04:07:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b68027740 1 --2- 172.21.15.97:0/1270111889 >> [v2:172.21.15.154:6833/3669418265,v1:172.21.15.154:6835/3669418265] conn(0x55b97be586f0 0x55b97be58ad0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b68027740 1 --2- 172.21.15.97:0/1270111889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55b97be8e450 0x55b97bea5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b3dffb640 0 client.4626 ms_handle_reset on v2:172.21.15.154:6833/3669418265 2024-03-29T04:07:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b68027740 1 --2- 172.21.15.97:0/1270111889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b97be86c40 0x55b97bea57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b68027740 1 --2- 172.21.15.97:0/1270111889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b97be86860 0x55b97bf175b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b68027740 1 -- 172.21.15.97:0/1270111889 >> 172.21.15.97:0/1270111889 conn(0x55b97be59d40 msgr2=0x55b97bec7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b68027740 1 -- 172.21.15.97:0/1270111889 shutdown_connections 2024-03-29T04:07:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b68027740 1 -- 172.21.15.97:0/1270111889 wait complete. 2024-03-29T04:07:53.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b68027740 20 client.4626 trim_cache size 0 max 16384 2024-03-29T04:07:53.725 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:07:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f", 2024-03-29T04:07:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-03-29T04:07:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4297, 2024-03-29T04:07:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-29T04:07:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-29T04:07:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-29T04:07:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 12.309975464000001, 2024-03-29T04:07:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 6, 2024-03-29T04:07:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 20, 2024-03-29T04:07:53.725 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 20, 2024-03-29T04:07:53.726 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 19.537034121000001 2024-03-29T04:07:53.726 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:07:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b65359640 1 -- 172.21.15.97:0/4189386293 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9b4c06c560 msgr2=0x7f9b4c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b65359640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9b4c06c560 0x7f9b4c06ea20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9b60162260 tx=0x7f9b50009040 comp rx=0 tx=0).stop 2024-03-29T04:07:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b65359640 1 -- 172.21.15.97:0/4189386293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b6015e6b0 msgr2=0x7f9b600ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b65359640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b6015e6b0 0x7f9b600ca930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b580097d0 tx=0x7f9b580046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b65359640 1 -- 172.21.15.97:0/4189386293 shutdown_connections 2024-03-29T04:07:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b65359640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9b4c06c560 0x7f9b4c06ea20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b65359640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b6015e6b0 0x7f9b600ca930 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b65359640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9b600d3030 0x7f9b600b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b65359640 1 --2- 172.21.15.97:0/4189386293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b600d2ba0 0x7f9b601549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b65359640 1 -- 172.21.15.97:0/4189386293 >> 172.21.15.97:0/4189386293 conn(0x7f9b6008bfc0 msgr2=0x7f9b600d1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b65359640 1 -- 172.21.15.97:0/4189386293 shutdown_connections 2024-03-29T04:07:53.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:53.736+0000 7f9b65359640 1 -- 172.21.15.97:0/4189386293 wait complete. 2024-03-29T04:07:53.728 INFO:teuthology.run_tasks:Running task sequential... 2024-03-29T04:07:53.849 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-29T04:07:53.849 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-29T04:07:53.856 INFO:teuthology.task.print:Enabling mgr modules 2024-03-29T04:07:53.856 INFO:teuthology.run_tasks:Running task kclient... 2024-03-29T04:07:53.881 INFO:tasks.kclient:Mounting kernel clients... 2024-03-29T04:07:53.881 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-29T04:07:53.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:53.881 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-29T04:07:53.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:53.897 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-03-29T04:07:53.965 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2024-03-29T04:07:53.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:07:53.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:53.967 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-29T04:07:54.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:54.023 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-03-29T04:07:54.090 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2024-03-29T04:07:54.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:07:54.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:54.091 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-29T04:07:54.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:54.107 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2024-03-29T04:07:54.175 INFO:teuthology.orchestra.run.smithi154.stderr:Cannot find device "ceph-brx" 2024-03-29T04:07:54.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:07:54.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:54.177 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-29T04:07:54.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:54.232 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2024-03-29T04:07:54.299 INFO:teuthology.orchestra.run.smithi154.stderr:Cannot find device "ceph-brx" 2024-03-29T04:07:54.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:07:54.302 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-29T04:07:54.302 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:07:54.302 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:07:54.302 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:07:54.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.423+0000 7ffbd9b00640 1 Processor -- start 2024-03-29T04:07:54.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.423+0000 7ffbd9b00640 1 -- start start 2024-03-29T04:07:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd9b00640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffbd4146030 0x7ffbd40d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd9b00640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffbd40d0bd0 0x7ffbd40cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd9b00640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbd40ce0b0 con 0x7ffbd40cd930 2024-03-29T04:07:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd9b00640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ffbd41611a0 con 0x7ffbd4146030 2024-03-29T04:07:54.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd9b00640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ffbd40d1030 con 0x7ffbd40d0bd0 2024-03-29T04:07:54.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd37fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffbd4146030 0x7ffbd40d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:54.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd2ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffbd40d0bd0 0x7ffbd40cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:54.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd2ffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffbd40d0bd0 0x7ffbd40cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:48890/0 (socket says 172.21.15.97:48890) 2024-03-29T04:07:54.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd3fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ffbd40cd930 0x7ffbd406e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:53992/0 (socket says 172.21.15.97:53992) 2024-03-29T04:07:54.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd37fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffbd4146030 0x7ffbd40d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:60874/0 (socket says 172.21.15.97:60874) 2024-03-29T04:07:54.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd2ffd640 1 -- 172.21.15.97:0/343201643 learned_addr learned my addr 172.21.15.97:0/343201643 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:54.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd37fe640 1 -- 172.21.15.97:0/343201643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffbd40d0bd0 msgr2=0x7ffbd40cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:54.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd37fe640 1 --2- 172.21.15.97:0/343201643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffbd40d0bd0 0x7ffbd40cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:54.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd37fe640 1 -- 172.21.15.97:0/343201643 >> v1:172.21.15.154:6789/0 conn(0x7ffbd40cd930 legacy=0x7ffbd406e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:54.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd37fe640 1 -- 172.21.15.97:0/343201643 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd41603e0 con 0x7ffbd4146030 2024-03-29T04:07:54.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.424+0000 7ffbd2ffd640 1 --2- 172.21.15.97:0/343201643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffbd40d0bd0 0x7ffbd40cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:07:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.425+0000 7ffbd37fe640 1 --2- 172.21.15.97:0/343201643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd4146030 0x7ffbd40d0690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40097a0 tx=0x7ffbc402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=991fa0c4f26bb08e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.425+0000 7ffbd27fc640 1 -- 172.21.15.97:0/343201643 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4035020 con 0x7ffbd4146030 2024-03-29T04:07:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.425+0000 7ffbd27fc640 1 -- 172.21.15.97:0/343201643 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc4031830 con 0x7ffbd4146030 2024-03-29T04:07:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.425+0000 7ffbd27fc640 1 -- 172.21.15.97:0/343201643 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4031c00 con 0x7ffbd4146030 2024-03-29T04:07:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.426+0000 7ffbd9b00640 1 -- 172.21.15.97:0/343201643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd4146030 msgr2=0x7ffbd40d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.426+0000 7ffbd9b00640 1 --2- 172.21.15.97:0/343201643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd4146030 0x7ffbd40d0690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40097a0 tx=0x7ffbc402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:07:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.426+0000 7ffbd9b00640 1 -- 172.21.15.97:0/343201643 shutdown_connections 2024-03-29T04:07:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.426+0000 7ffbd9b00640 1 --2- 172.21.15.97:0/343201643 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffbd40d0bd0 0x7ffbd40cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.426+0000 7ffbd9b00640 1 --2- 172.21.15.97:0/343201643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd4146030 0x7ffbd40d0690 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:54.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.426+0000 7ffbd9b00640 1 -- 172.21.15.97:0/343201643 >> 172.21.15.97:0/343201643 conn(0x7ffbd406b510 msgr2=0x7ffbd406b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.426+0000 7ffbd9b00640 1 -- 172.21.15.97:0/343201643 shutdown_connections 2024-03-29T04:07:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.426+0000 7ffbd9b00640 1 -- 172.21.15.97:0/343201643 wait complete. 2024-03-29T04:07:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.427+0000 7ffbd9b00640 1 Processor -- start 2024-03-29T04:07:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.427+0000 7ffbd9b00640 1 -- start start 2024-03-29T04:07:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd9b00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbd40cd930 0x7ffbd413a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd9b00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd40d0bd0 0x7ffbd413c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd9b00640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffbd4146030 0x7ffbd413c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd9b00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd40cddc0 con 0x7ffbd40cd930 2024-03-29T04:07:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd9b00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41609a0 con 0x7ffbd40d0bd0 2024-03-29T04:07:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd9b00640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ffbd4146410 con 0x7ffbd4146030 2024-03-29T04:07:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbd40cd930 0x7ffbd413a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:54.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbd40cd930 0x7ffbd413a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52204/0 (socket says 172.21.15.97:52204) 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd37fe640 1 -- 172.21.15.97:0/555882931 learned_addr learned my addr 172.21.15.97:0/555882931 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd3fff640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffbd4146030 0x7ffbd413c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd2ffd640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd40d0bd0 0x7ffbd413c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd37fe640 1 -- 172.21.15.97:0/555882931 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffbd4146030 msgr2=0x7ffbd413c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd37fe640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffbd4146030 0x7ffbd413c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd37fe640 1 -- 172.21.15.97:0/555882931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd40d0bd0 msgr2=0x7ffbd413c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd37fe640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd40d0bd0 0x7ffbd413c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd37fe640 1 -- 172.21.15.97:0/555882931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd413e090 con 0x7ffbd40cd930 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.428+0000 7ffbd37fe640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbd40cd930 0x7ffbd413a1a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffbc402b910 tx=0x7ffbc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.429+0000 7ffbbbfff640 1 -- 172.21.15.97:0/555882931 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4040070 con 0x7ffbd40cd930 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.429+0000 7ffbbbfff640 1 -- 172.21.15.97:0/555882931 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc403a930 con 0x7ffbd40cd930 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.429+0000 7ffbd9b00640 1 -- 172.21.15.97:0/555882931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd413e370 con 0x7ffbd40cd930 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.429+0000 7ffbbbfff640 1 -- 172.21.15.97:0/555882931 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4035040 con 0x7ffbd40cd930 2024-03-29T04:07:54.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.429+0000 7ffbd9b00640 1 -- 172.21.15.97:0/555882931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd413e930 con 0x7ffbd40cd930 2024-03-29T04:07:54.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.430+0000 7ffbd9b00640 1 -- 172.21.15.97:0/555882931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd4068640 con 0x7ffbd40cd930 2024-03-29T04:07:54.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.430+0000 7ffbbbfff640 1 -- 172.21.15.97:0/555882931 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ffbc404b060 con 0x7ffbd40cd930 2024-03-29T04:07:54.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.430+0000 7ffbbbfff640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ffbc006c530 0x7ffbc006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:54.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.430+0000 7ffbbbfff640 1 -- 172.21.15.97:0/555882931 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffbc40acdc0 con 0x7ffbd40cd930 2024-03-29T04:07:54.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.431+0000 7ffbd2ffd640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ffbc006c530 0x7ffbc006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:54.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.431+0000 7ffbd2ffd640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ffbc006c530 0x7ffbc006e9f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffbd41f7e40 tx=0x7ffbc8009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:54.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.435+0000 7ffbbbfff640 1 -- 172.21.15.97:0/555882931 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ffbc4031800 con 0x7ffbd40cd930 2024-03-29T04:07:54.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.651+0000 7ffbd9b00640 1 -- 172.21.15.97:0/555882931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffbd40d1370 con 0x7ffbd40cd930 2024-03-29T04:07:54.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.651+0000 7ffbbbfff640 1 -- 172.21.15.97:0/555882931 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7ffbc407ad90 con 0x7ffbd40cd930 2024-03-29T04:07:54.631 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T04:07:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.653+0000 7ffbd9b00640 1 -- 172.21.15.97:0/555882931 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ffbc006c530 msgr2=0x7ffbc006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.653+0000 7ffbd9b00640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ffbc006c530 0x7ffbc006e9f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffbd41f7e40 tx=0x7ffbc8009240 comp rx=0 tx=0).stop 2024-03-29T04:07:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.653+0000 7ffbd9b00640 1 -- 172.21.15.97:0/555882931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbd40cd930 msgr2=0x7ffbd413a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.653+0000 7ffbd9b00640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbd40cd930 0x7ffbd413a1a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffbc402b910 tx=0x7ffbc40046c0 comp rx=0 tx=0).stop 2024-03-29T04:07:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.653+0000 7ffbd9b00640 1 -- 172.21.15.97:0/555882931 shutdown_connections 2024-03-29T04:07:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.653+0000 7ffbd9b00640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ffbc006c530 0x7ffbc006e9f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.653+0000 7ffbd9b00640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffbd4146030 0x7ffbd413c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.653+0000 7ffbd9b00640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd40d0bd0 0x7ffbd413c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.653+0000 7ffbd9b00640 1 --2- 172.21.15.97:0/555882931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbd40cd930 0x7ffbd413a1a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.653+0000 7ffbd9b00640 1 -- 172.21.15.97:0/555882931 >> 172.21.15.97:0/555882931 conn(0x7ffbd406b510 msgr2=0x7ffbd415d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.653+0000 7ffbd9b00640 1 -- 172.21.15.97:0/555882931 shutdown_connections 2024-03-29T04:07:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54.653+0000 7ffbd9b00640 1 -- 172.21.15.97:0/555882931 wait complete. 2024-03-29T04:07:54.644 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:07:54.644 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:07:54.644 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-03-29T04:07:54.644 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T04:07:54.644 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:07:54.644 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:07:54.644 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:07:54.644 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:07:54.644 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:07:54.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:54.644 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-03-29T04:07:54.659 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:07:54.660 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:07:54.661 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:07:54.661 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:07:54.661 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42695sec preferred_lft 42695sec 2024-03-29T04:07:54.661 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-03-29T04:07:54.661 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:54.661 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T04:07:54.661 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:54.661 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-29T04:07:54.662 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2024-03-29T04:07:54.662 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2024-03-29T04:07:54.662 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2024-03-29T04:07:54.662 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T04:07:54.662 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-29T04:07:54.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:54.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:54.858 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:07:54.886 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-03-29T04:07:54.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:54.887 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-03-29T04:07:54.949 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-03-29T04:07:54.949 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:07:54.949 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:07:54.949 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:07:54.949 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:07:54.950 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:54.950 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-29T04:07:54.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:07:54.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:07:54.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:07:54.950 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-29T04:07:55.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:55.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:55.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:55.257 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-29T04:07:55.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:55.272 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-03-29T04:07:55.327 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:55.327 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-29T04:07:55.327 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:07:55.327 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-29T04:07:55.327 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-29T04:07:55.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:55.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:55.449 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-29T04:07:55.449 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:55.449 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-29T04:07:55.449 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-29T04:07:55.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T04:07:55.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-29T04:07:55.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-29T04:07:55.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-29T04:07:55.450 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-29T04:07:55.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:55.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:55.582 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:55.582 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-29T04:07:55.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-03-29T04:07:55.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T04:07:55.582 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-29T04:07:55.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:55.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:55.664 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:07:55.664 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T04:07:55.678 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:07:55.678 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:07:55.679 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:07:55.733 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-29T04:07:55.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:55.733 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:07:55.817 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:07:55.817 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:07:55.935 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:07:55.937 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:07:55.937 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.154:3300,172.21.15.154:3301" 2024-03-29T04:07:55.937 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f.cephfs=/ 2024-03-29T04:07:55.937 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.154:3300/172.21.15.154:3301" will pass to kernel 2024-03-29T04:07:55.939 INFO:teuthology.orchestra.run.smithi097.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-29T04:07:55.939 INFO:teuthology.orchestra.run.smithi097.stdout: You just mounted a file system that supports labels which does not 2024-03-29T04:07:55.939 INFO:teuthology.orchestra.run.smithi097.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-29T04:07:55.939 INFO:teuthology.orchestra.run.smithi097.stdout: applications will generate AVC messages and not be allowed access to 2024-03-29T04:07:55.939 INFO:teuthology.orchestra.run.smithi097.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-29T04:07:55.939 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-29T04:07:55.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:55.939 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:07:55.966 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:07:56.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:07:56.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:56.066 INFO:teuthology.orchestra.run.smithi097.stdout:client4693 2024-03-29T04:07:56.067 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:56.067 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:07:56.082 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:07:56.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:07:56.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:56.182 INFO:teuthology.orchestra.run.smithi097.stdout:client4693 2024-03-29T04:07:56.182 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:07:56.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:07:56.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:56.242 INFO:teuthology.orchestra.run.smithi097.stdout:client4693 2024-03-29T04:07:56.242 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f.client4693/status') 2024-03-29T04:07:56.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:56.307 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.4693 (3)172.21.15.97:0/228495629 2024-03-29T04:07:56.307 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2024-03-29T04:07:56.307 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2024-03-29T04:07:56.307 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2024-03-29T04:07:56.308 INFO:teuthology.orchestra.run.smithi097.stderr:69 bytes copied, 2.0653e-05 s, 3.3 MB/s 2024-03-29T04:07:56.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:56.314 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-03-29T04:07:56.314 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:07:56.315 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-29T04:07:56.315 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:07:56.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.436+0000 7fb1b1524640 1 Processor -- start 2024-03-29T04:07:56.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.436+0000 7fb1b1524640 1 -- start start 2024-03-29T04:07:56.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.437+0000 7fb1b1524640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb1ac15e9b0 0x7fb1ac0d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:56.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.437+0000 7fb1b1524640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb1ac0d4f30 0x7fb1ac162f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:56.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.437+0000 7fb1b1524640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1ac0d54f0 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.437+0000 7fb1b1524640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fb1ac1634b0 con 0x7fb1ac0d4f30 2024-03-29T04:07:56.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.437+0000 7fb1b1524640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fb1ac0cfcf0 con 0x7fb1ac15e9b0 2024-03-29T04:07:56.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.437+0000 7fb1a27fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb1ac0d4f30 0x7fb1ac162f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:56.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.437+0000 7fb1a27fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb1ac0d4f30 0x7fb1ac162f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52218/0 (socket says 172.21.15.97:52218) 2024-03-29T04:07:56.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.437+0000 7fb1a27fc640 1 -- 172.21.15.97:0/1603678015 learned_addr learned my addr 172.21.15.97:0/1603678015 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:56.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.437+0000 7fb1aaffd640 1 --2- 172.21.15.97:0/1603678015 >> v2:172.21.15.154:3300/0 conn(0x7fb1ac15e9b0 0x7fb1ac0d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:56.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.437+0000 7fb1a27fc640 1 -- 172.21.15.97:0/1603678015 >> v2:172.21.15.154:3300/0 conn(0x7fb1ac15e9b0 msgr2=0x7fb1ac0d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:07:56.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.437+0000 7fb1a27fc640 1 --2- 172.21.15.97:0/1603678015 >> v2:172.21.15.154:3300/0 conn(0x7fb1ac15e9b0 0x7fb1ac0d0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:56.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.437+0000 7fb1a27fc640 1 -- 172.21.15.97:0/1603678015 >> v1:172.21.15.154:6789/0 conn(0x7fb1ac0d2bf0 legacy=0x7fb1ac06c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:56.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.437+0000 7fb1a27fc640 1 -- 172.21.15.97:0/1603678015 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac0cf3c0 con 0x7fb1ac0d4f30 2024-03-29T04:07:56.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.438+0000 7fb1a27fc640 1 --2- 172.21.15.97:0/1603678015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac0d4f30 0x7fb1ac162f70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac08cc40 tx=0x7fb19402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bdce34f16619bf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:56.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.438+0000 7fb1aa7fc640 1 -- 172.21.15.97:0/1603678015 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194034070 con 0x7fb1ac0d4f30 2024-03-29T04:07:56.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.438+0000 7fb1aa7fc640 1 -- 172.21.15.97:0/1603678015 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb194030860 con 0x7fb1ac0d4f30 2024-03-29T04:07:56.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.438+0000 7fb1aa7fc640 1 -- 172.21.15.97:0/1603678015 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194030c30 con 0x7fb1ac0d4f30 2024-03-29T04:07:56.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.439+0000 7fb1b1524640 1 -- 172.21.15.97:0/1603678015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac0d4f30 msgr2=0x7fb1ac162f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:56.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.439+0000 7fb1b1524640 1 --2- 172.21.15.97:0/1603678015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac0d4f30 0x7fb1ac162f70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac08cc40 tx=0x7fb19402b9a0 comp rx=0 tx=0).stop 2024-03-29T04:07:56.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.439+0000 7fb1b1524640 1 -- 172.21.15.97:0/1603678015 shutdown_connections 2024-03-29T04:07:56.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.439+0000 7fb1b1524640 1 --2- 172.21.15.97:0/1603678015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac0d4f30 0x7fb1ac162f70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:56.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.439+0000 7fb1b1524640 1 --2- 172.21.15.97:0/1603678015 >> v2:172.21.15.154:3300/0 conn(0x7fb1ac15e9b0 0x7fb1ac0d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:56.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.439+0000 7fb1b1524640 1 -- 172.21.15.97:0/1603678015 >> 172.21.15.97:0/1603678015 conn(0x7fb1ac08bfc0 msgr2=0x7fb1ac0d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:56.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.439+0000 7fb1b1524640 1 -- 172.21.15.97:0/1603678015 shutdown_connections 2024-03-29T04:07:56.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.439+0000 7fb1b1524640 1 -- 172.21.15.97:0/1603678015 wait complete. 2024-03-29T04:07:56.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.440+0000 7fb1b1524640 1 Processor -- start 2024-03-29T04:07:56.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.440+0000 7fb1b1524640 1 -- start start 2024-03-29T04:07:56.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1b1524640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac0d2bf0 0x7fb1ac0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:56.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1b1524640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1ac0d4f30 0x7fb1ac1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:56.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1b1524640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1ac15e9b0 0x7fb1ac0b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:56.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1b1524640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac15ed90 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1b1524640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ac0cf980 con 0x7fb1ac15e9b0 2024-03-29T04:07:56.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1b1524640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb1ac0d3060 con 0x7fb1ac0d4f30 2024-03-29T04:07:56.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1aaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac0d2bf0 0x7fb1ac0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:56.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1aaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac0d2bf0 0x7fb1ac0ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52226/0 (socket says 172.21.15.97:52226) 2024-03-29T04:07:56.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1aaffd640 1 -- 172.21.15.97:0/3575803178 learned_addr learned my addr 172.21.15.97:0/3575803178 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:56.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1ab7fe640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1ac15e9b0 0x7fb1ac0b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:56.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1a27fc640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1ac0d4f30 0x7fb1ac1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:56.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1aaffd640 1 -- 172.21.15.97:0/3575803178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1ac0d4f30 msgr2=0x7fb1ac1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:56.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1aaffd640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1ac0d4f30 0x7fb1ac1554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:56.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1aaffd640 1 -- 172.21.15.97:0/3575803178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1ac15e9b0 msgr2=0x7fb1ac0b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1aaffd640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1ac15e9b0 0x7fb1ac0b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1aaffd640 1 -- 172.21.15.97:0/3575803178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac155ef0 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.441+0000 7fb1aaffd640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac0d2bf0 0x7fb1ac0ce210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00e8a0 tx=0x7fb19c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.442+0000 7fb1a3fff640 1 -- 172.21.15.97:0/3575803178 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c017070 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.442+0000 7fb1a3fff640 1 -- 172.21.15.97:0/3575803178 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb19c00c940 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.442+0000 7fb1b1524640 1 -- 172.21.15.97:0/3575803178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ac0b9970 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.442+0000 7fb1a3fff640 1 -- 172.21.15.97:0/3575803178 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00cd30 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.442+0000 7fb1b1524640 1 -- 172.21.15.97:0/3575803178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ac0b9f30 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.443+0000 7fb1b1524640 1 -- 172.21.15.97:0/3575803178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ac068640 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.443+0000 7fb1a3fff640 1 -- 172.21.15.97:0/3575803178 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb19c01a070 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.443+0000 7fb1a3fff640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb18c06c530 0x7fb18c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.443+0000 7fb1a3fff640 1 -- 172.21.15.97:0/3575803178 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb19c08b850 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.443+0000 7fb1a27fc640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb18c06c530 0x7fb18c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:56.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.444+0000 7fb1a27fc640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb18c06c530 0x7fb18c06e9f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb194036000 tx=0x7fb194008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:56.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.448+0000 7fb1a3fff640 1 -- 172.21.15.97:0/3575803178 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb19c059820 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.662+0000 7fb1b1524640 1 -- 172.21.15.97:0/3575803178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb1ac0d29d0 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.662+0000 7fb1a3fff640 1 -- 172.21.15.97:0/3575803178 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fb19c05db20 con 0x7fb1ac0d2bf0 2024-03-29T04:07:56.642 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T04:07:56.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.664+0000 7fb1b1524640 1 -- 172.21.15.97:0/3575803178 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb18c06c530 msgr2=0x7fb18c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:56.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.664+0000 7fb1b1524640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb18c06c530 0x7fb18c06e9f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb194036000 tx=0x7fb194008030 comp rx=0 tx=0).stop 2024-03-29T04:07:56.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.664+0000 7fb1b1524640 1 -- 172.21.15.97:0/3575803178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac0d2bf0 msgr2=0x7fb1ac0ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:56.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.664+0000 7fb1b1524640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac0d2bf0 0x7fb1ac0ce210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00e8a0 tx=0x7fb19c00ed60 comp rx=0 tx=0).stop 2024-03-29T04:07:56.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.664+0000 7fb1b1524640 1 -- 172.21.15.97:0/3575803178 shutdown_connections 2024-03-29T04:07:56.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.664+0000 7fb1b1524640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb18c06c530 0x7fb18c06e9f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:56.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.664+0000 7fb1b1524640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1ac15e9b0 0x7fb1ac0b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:56.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.664+0000 7fb1b1524640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1ac0d4f30 0x7fb1ac1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:56.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.664+0000 7fb1b1524640 1 --2- 172.21.15.97:0/3575803178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1ac0d2bf0 0x7fb1ac0ce210 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:56.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.664+0000 7fb1b1524640 1 -- 172.21.15.97:0/3575803178 >> 172.21.15.97:0/3575803178 conn(0x7fb1ac08bfc0 msgr2=0x7fb1ac160f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:56.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.665+0000 7fb1b1524640 1 -- 172.21.15.97:0/3575803178 shutdown_connections 2024-03-29T04:07:56.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56.665+0000 7fb1b1524640 1 -- 172.21.15.97:0/3575803178 wait complete. 2024-03-29T04:07:56.655 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:07:56.655 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:07:56.655 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-03-29T04:07:56.655 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-29T04:07:56.655 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-29T04:07:56.655 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:07:56.655 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:07:56.655 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:07:56.656 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-29T04:07:56.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:56.656 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-03-29T04:07:56.671 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:07:56.671 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:07:56.671 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:07:56.671 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:56.671 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-03-29T04:07:56.671 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:56.671 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:07:56.671 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-03-29T04:07:56.672 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-03-29T04:07:56.672 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:07:56.672 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-03-29T04:07:56.672 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42693sec preferred_lft 42693sec 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:4d:74:f4:86:4e brd ff:ff:ff:ff:ff:ff 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::dccd:95ff:fefa:4739/64 scope link 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:56.673 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:07:56.674 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:4d:74:f4:86:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:07:56.674 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c4d:74ff:fef4:864e/64 scope link tentative 2024-03-29T04:07:56.674 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:56.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:56.674 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:07:56.737 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-03-29T04:07:56.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:56.739 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-03-29T04:07:56.794 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-03-29T04:07:56.794 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:07:56.794 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:07:56.794 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:07:56.794 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:07:56.795 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:56.795 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-29T04:07:56.795 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:07:56.795 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:07:56.795 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:07:56.795 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-29T04:07:56.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:56.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:56.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:56.929 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-29T04:07:56.944 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T04:07:56.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:56.944 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-03-29T04:07:56.999 INFO:teuthology.orchestra.run.smithi097.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-29T04:07:57.000 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:57.000 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-29T04:07:57.000 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:07:57.000 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-29T04:07:57.000 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-29T04:07:57.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:57.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:57.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:57.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-29T04:07:57.147 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:07:57.147 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:07:57.147 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:07:57.148 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:57.148 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-03-29T04:07:57.148 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:57.148 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:07:57.148 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 72:f9:87:d0:61:e2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:07:57.148 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:07:57.148 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:57.148 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::70f9:87ff:fed0:61e2/64 scope link tentative 2024-03-29T04:07:57.148 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:57.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:57.149 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-29T04:07:57.217 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:07:57.217 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:07:57.217 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:07:57.217 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:57.217 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-03-29T04:07:57.217 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:57.217 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:07:57.217 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 72:f9:87:d0:61:e2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:07:57.217 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:07:57.218 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:57.218 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::70f9:87ff:fed0:61e2/64 scope link tentative 2024-03-29T04:07:57.218 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:57.219 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-29T04:07:57.219 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:57.219 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-29T04:07:57.219 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-29T04:07:57.219 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-29T04:07:57.219 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-29T04:07:57.219 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-29T04:07:57.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-29T04:07:57.220 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-29T04:07:57.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:57.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:57.383 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:57.383 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-29T04:07:57.383 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.1 up 2024-03-29T04:07:57.383 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.1 master ceph-brx 2024-03-29T04:07:57.384 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-29T04:07:57.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:57.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:57.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:07:57.467 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-29T04:07:57.481 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-29T04:07:57.481 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:07:57.482 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-29T04:07:57.536 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-03-29T04:07:57.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:57.536 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:07:57.621 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:07:57.621 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:07:57.640 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:07:57.640 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:07:57.640 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.154:3300,172.21.15.154:3301" 2024-03-29T04:07:57.640 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 2@51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f.cephfs=/ 2024-03-29T04:07:57.641 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.97:3300/172.21.15.154:3300/172.21.15.154:3301" will pass to kernel 2024-03-29T04:07:57.641 INFO:teuthology.orchestra.run.smithi097.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-29T04:07:57.641 INFO:teuthology.orchestra.run.smithi097.stdout: You just mounted a file system that supports labels which does not 2024-03-29T04:07:57.641 INFO:teuthology.orchestra.run.smithi097.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-29T04:07:57.641 INFO:teuthology.orchestra.run.smithi097.stdout: applications will generate AVC messages and not be allowed access to 2024-03-29T04:07:57.641 INFO:teuthology.orchestra.run.smithi097.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-29T04:07:57.642 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-29T04:07:57.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:57.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-29T04:07:57.669 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:07:57.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:07:57.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:57.765 INFO:teuthology.orchestra.run.smithi097.stdout:client4526 2024-03-29T04:07:57.765 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:07:57.766 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:07:57.781 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:07:57.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:07:57.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:57.877 INFO:teuthology.orchestra.run.smithi097.stdout:client4526 2024-03-29T04:07:57.877 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:07:57.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:07:57.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:57.932 INFO:teuthology.orchestra.run.smithi097.stdout:client4526 2024-03-29T04:07:57.932 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f.client4526/status') 2024-03-29T04:07:57.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:57.993 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.4526 (3)172.21.15.97:0/1273152657 2024-03-29T04:07:57.993 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2024-03-29T04:07:57.993 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2024-03-29T04:07:57.993 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2024-03-29T04:07:57.993 INFO:teuthology.orchestra.run.smithi097.stderr:70 bytes copied, 1.9831e-05 s, 3.5 MB/s 2024-03-29T04:07:57.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:58.000 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-29T04:07:58.001 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:07:58.001 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:07:58.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:07:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.120+0000 7f47d7d90640 1 Processor -- start 2024-03-29T04:07:58.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.120+0000 7f47d7d90640 1 -- start start 2024-03-29T04:07:58.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.121+0000 7f47d7d90640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47d0162ab0 0x7f47d00b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:58.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.121+0000 7f47d7d90640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d00d1a10 con 0x7f47d009b6e0 2024-03-29T04:07:58.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.121+0000 7f47d7d90640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d00d0e90 con 0x7f47d015e6e0 2024-03-29T04:07:58.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.121+0000 7f47d7d90640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f47d009bb40 con 0x7f47d0162ab0 2024-03-29T04:07:58.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.121+0000 7f47d632d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47d0162ab0 0x7f47d00b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:58.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.121+0000 7f47d532b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f47d009b6e0 0x7f47d01623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35686/0 (socket says 172.21.15.97:35686) 2024-03-29T04:07:58.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.121+0000 7f47d532b640 1 -- 172.21.15.97:0/3916726364 learned_addr learned my addr 172.21.15.97:0/3916726364 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:07:58.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.121+0000 7f47d632d640 1 -- 172.21.15.97:0/3916726364 >> v1:172.21.15.154:6789/0 conn(0x7f47d015e6e0 legacy=0x7f47d00d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:58.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.122+0000 7f47d632d640 1 -- 172.21.15.97:0/3916726364 >> v1:172.21.15.97:6789/0 conn(0x7f47d009b6e0 legacy=0x7f47d01623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:58.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.122+0000 7f47d4b2a640 1 -- 172.21.15.97:0/3916726364 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1006591487 0 0) 0x7f47d00d1a10 con 0x7f47d009b6e0 2024-03-29T04:07:58.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.122+0000 7f47d632d640 1 -- 172.21.15.97:0/3916726364 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d00d1170 con 0x7f47d0162ab0 2024-03-29T04:07:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.122+0000 7f47d632d640 1 --2- 172.21.15.97:0/3916726364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d0162ab0 0x7f47d00b30e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009d90 tx=0x7f47c802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae50ce35bcf77af server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.122+0000 7f47d4b2a640 1 -- 172.21.15.97:0/3916726364 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8039070 con 0x7f47d0162ab0 2024-03-29T04:07:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.122+0000 7f47d4b2a640 1 -- 172.21.15.97:0/3916726364 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c802cd00 con 0x7f47d0162ab0 2024-03-29T04:07:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.122+0000 7f47d4b2a640 1 -- 172.21.15.97:0/3916726364 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8034660 con 0x7f47d0162ab0 2024-03-29T04:07:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.123+0000 7f47d7d90640 1 -- 172.21.15.97:0/3916726364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d0162ab0 msgr2=0x7f47d00b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.123+0000 7f47d7d90640 1 --2- 172.21.15.97:0/3916726364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d0162ab0 0x7f47d00b30e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009d90 tx=0x7f47c802bf50 comp rx=0 tx=0).stop 2024-03-29T04:07:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.123+0000 7f47d7d90640 1 -- 172.21.15.97:0/3916726364 shutdown_connections 2024-03-29T04:07:58.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.123+0000 7f47d7d90640 1 --2- 172.21.15.97:0/3916726364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d0162ab0 0x7f47d00b30e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.123+0000 7f47d7d90640 1 -- 172.21.15.97:0/3916726364 >> 172.21.15.97:0/3916726364 conn(0x7f47d00b3f10 msgr2=0x7f47d00b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.123+0000 7f47d7d90640 1 -- 172.21.15.97:0/3916726364 shutdown_connections 2024-03-29T04:07:58.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.123+0000 7f47d7d90640 1 -- 172.21.15.97:0/3916726364 wait complete. 2024-03-29T04:07:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.124+0000 7f47d7d90640 1 Processor -- start 2024-03-29T04:07:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.124+0000 7f47d7d90640 1 -- start start 2024-03-29T04:07:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d7d90640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47d009b6e0 0x7f47d00b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d7d90640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d015e6e0 0x7f47d0152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d7d90640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47d0162ab0 0x7f47d0161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d7d90640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00b1d80 con 0x7f47d015e6e0 2024-03-29T04:07:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d7d90640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00d1730 con 0x7f47d0162ab0 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d7d90640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f47d006d910 con 0x7f47d009b6e0 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d532b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d015e6e0 0x7f47d0152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d532b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d015e6e0 0x7f47d0152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52236/0 (socket says 172.21.15.97:52236) 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d532b640 1 -- 172.21.15.97:0/2523633478 learned_addr learned my addr 172.21.15.97:0/2523633478 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d632d640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47d0162ab0 0x7f47d0161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d5b2c640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47d009b6e0 0x7f47d00b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d532b640 1 -- 172.21.15.97:0/2523633478 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47d009b6e0 msgr2=0x7f47d00b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d532b640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47d009b6e0 0x7f47d00b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d532b640 1 -- 172.21.15.97:0/2523633478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47d0162ab0 msgr2=0x7f47d0161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d532b640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47d0162ab0 0x7f47d0161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.125+0000 7f47d532b640 1 -- 172.21.15.97:0/2523633478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d0153090 con 0x7f47d015e6e0 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.126+0000 7f47d532b640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d015e6e0 0x7f47d0152b50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f47c000c420 tx=0x7f47c000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.126+0000 7f47c67fc640 1 -- 172.21.15.97:0/2523633478 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c00074e0 con 0x7f47d015e6e0 2024-03-29T04:07:58.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.126+0000 7f47c67fc640 1 -- 172.21.15.97:0/2523633478 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c00047e0 con 0x7f47d015e6e0 2024-03-29T04:07:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.126+0000 7f47d7d90640 1 -- 172.21.15.97:0/2523633478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d01603e0 con 0x7f47d015e6e0 2024-03-29T04:07:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.126+0000 7f47c67fc640 1 -- 172.21.15.97:0/2523633478 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0004c10 con 0x7f47d015e6e0 2024-03-29T04:07:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.126+0000 7f47d7d90640 1 -- 172.21.15.97:0/2523633478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d01609a0 con 0x7f47d015e6e0 2024-03-29T04:07:58.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.127+0000 7f47d7d90640 1 -- 172.21.15.97:0/2523633478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d0068640 con 0x7f47d015e6e0 2024-03-29T04:07:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.127+0000 7f47c67fc640 1 -- 172.21.15.97:0/2523633478 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f47c0011070 con 0x7f47d015e6e0 2024-03-29T04:07:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.128+0000 7f47c67fc640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f47b406c530 0x7f47b406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:07:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.128+0000 7f47c67fc640 1 -- 172.21.15.97:0/2523633478 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f47c008b0c0 con 0x7f47d015e6e0 2024-03-29T04:07:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.128+0000 7f47d5b2c640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f47b406c530 0x7f47b406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:07:58.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.128+0000 7f47d5b2c640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f47b406c530 0x7f47b406e9f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f47d00d0350 tx=0x7f47cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:07:58.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.132+0000 7f47c67fc640 1 -- 172.21.15.97:0/2523633478 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f47c0059120 con 0x7f47d015e6e0 2024-03-29T04:07:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.346+0000 7f47d7d90640 1 -- 172.21.15.97:0/2523633478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f47d00d1e40 con 0x7f47d015e6e0 2024-03-29T04:07:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.347+0000 7f47c67fc640 1 -- 172.21.15.97:0/2523633478 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f47c005d420 con 0x7f47d015e6e0 2024-03-29T04:07:58.327 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T04:07:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.349+0000 7f47d7d90640 1 -- 172.21.15.97:0/2523633478 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f47b406c530 msgr2=0x7f47b406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.349+0000 7f47d7d90640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f47b406c530 0x7f47b406e9f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f47d00d0350 tx=0x7f47cc009040 comp rx=0 tx=0).stop 2024-03-29T04:07:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.349+0000 7f47d7d90640 1 -- 172.21.15.97:0/2523633478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d015e6e0 msgr2=0x7f47d0152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:07:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.349+0000 7f47d7d90640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d015e6e0 0x7f47d0152b50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f47c000c420 tx=0x7f47c000c8e0 comp rx=0 tx=0).stop 2024-03-29T04:07:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.349+0000 7f47d7d90640 1 -- 172.21.15.97:0/2523633478 shutdown_connections 2024-03-29T04:07:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.349+0000 7f47d7d90640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f47b406c530 0x7f47b406e9f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.349+0000 7f47d7d90640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47d0162ab0 0x7f47d0161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.349+0000 7f47d7d90640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47d015e6e0 0x7f47d0152b50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.349+0000 7f47d7d90640 1 --2- 172.21.15.97:0/2523633478 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47d009b6e0 0x7f47d00b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:07:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.349+0000 7f47d7d90640 1 -- 172.21.15.97:0/2523633478 >> 172.21.15.97:0/2523633478 conn(0x7f47d00b3f10 msgr2=0x7f47d00ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:07:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.349+0000 7f47d7d90640 1 -- 172.21.15.97:0/2523633478 shutdown_connections 2024-03-29T04:07:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:07:58.349+0000 7f47d7d90640 1 -- 172.21.15.97:0/2523633478 wait complete. 2024-03-29T04:07:58.339 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:07:58.339 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:07:58.340 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-03-29T04:07:58.340 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-29T04:07:58.340 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:07:58.340 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:07:58.340 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:07:58.340 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:07:58.340 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:07:58.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:58.340 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-03-29T04:07:58.356 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:07:58.356 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:07:58.356 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:07:58.356 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:58.356 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-03-29T04:07:58.356 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42706sec preferred_lft 42706sec 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2024-03-29T04:07:58.357 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:07:58.358 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T04:07:58.358 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:58.358 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-29T04:07:58.358 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add name ceph-brx type bridge 2024-03-29T04:07:58.358 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr flush dev ceph-brx 2024-03-29T04:07:58.358 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set ceph-brx up 2024-03-29T04:07:58.358 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T04:07:58.358 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-29T04:07:58.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:58.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:58.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:58.663 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:07:58.690 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-03-29T04:07:58.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:58.692 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-03-29T04:07:58.757 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-03-29T04:07:58.757 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:07:58.757 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:07:58.757 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:07:58.757 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:07:58.758 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:58.758 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-29T04:07:58.758 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:07:58.758 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:07:58.758 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:07:58.758 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-29T04:07:58.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:59.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:59.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:59.038 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-29T04:07:59.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:59.053 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list-id 2024-03-29T04:07:59.107 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:59.107 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-29T04:07:59.107 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:07:59.107 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-29T04:07:59.108 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-29T04:07:59.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:59.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:59.230 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-29T04:07:59.230 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:59.230 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-29T04:07:59.230 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-29T04:07:59.230 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T04:07:59.230 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-29T04:07:59.230 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-29T04:07:59.230 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-29T04:07:59.231 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-29T04:07:59.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:59.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:59.378 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:07:59.378 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-29T04:07:59.378 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2024-03-29T04:07:59.379 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T04:07:59.379 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-29T04:07:59.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:07:59.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:59.480 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:07:59.480 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-29T04:07:59.494 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:07:59.494 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:07:59.494 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:07:59.549 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-29T04:07:59.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:59.591 DEBUG:teuthology.orchestra.run.smithi154:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:07:59.640 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:07:59.640 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:07:59.738 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:07:59.738 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:07:59.738 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.154:3300,172.21.15.154:3301" 2024-03-29T04:07:59.738 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: 1@51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f.cephfs=/ 2024-03-29T04:07:59.738 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.154:3300/172.21.15.154:3301" will pass to kernel 2024-03-29T04:07:59.739 INFO:teuthology.orchestra.run.smithi154.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-29T04:07:59.739 INFO:teuthology.orchestra.run.smithi154.stdout: You just mounted a file system that supports labels which does not 2024-03-29T04:07:59.739 INFO:teuthology.orchestra.run.smithi154.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-29T04:07:59.740 INFO:teuthology.orchestra.run.smithi154.stdout: applications will generate AVC messages and not be allowed access to 2024-03-29T04:07:59.740 INFO:teuthology.orchestra.run.smithi154.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-29T04:07:59.741 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-29T04:07:59.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:07:59.741 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:07:59.768 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:07:59.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:07:59.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:59.863 INFO:teuthology.orchestra.run.smithi154.stdout:client4532 2024-03-29T04:07:59.863 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:07:59.863 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:07:59.878 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:07:59.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:07:59.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:07:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:07:59.973 INFO:teuthology.orchestra.run.smithi154.stdout:client4532 2024-03-29T04:07:59.974 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:08:00.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:08:00.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:00.030 INFO:teuthology.orchestra.run.smithi154.stdout:client4532 2024-03-29T04:08:00.031 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f.client4532/status') 2024-03-29T04:08:00.078 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:08:00.091 INFO:teuthology.orchestra.run.smithi154.stdout:instance: client.4532 (3)192.168.0.1:0/3994826782 2024-03-29T04:08:00.091 INFO:teuthology.orchestra.run.smithi154.stdout:blocklisted: false 2024-03-29T04:08:00.091 INFO:teuthology.orchestra.run.smithi154.stderr:0+1 records in 2024-03-29T04:08:00.091 INFO:teuthology.orchestra.run.smithi154.stderr:0+1 records out 2024-03-29T04:08:00.092 INFO:teuthology.orchestra.run.smithi154.stderr:69 bytes copied, 2.051e-05 s, 3.4 MB/s 2024-03-29T04:08:00.092 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:00.098 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-03-29T04:08:00.098 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:08:00.098 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-29T04:08:00.098 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:08:00.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.230+0000 7f7699dd4640 1 Processor -- start 2024-03-29T04:08:00.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.230+0000 7f7699dd4640 1 -- start start 2024-03-29T04:08:00.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f7699dd4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f769415ec60 0x7f76940ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:00.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f7699dd4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f76940b4340 0x7f76940d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:00.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f7699dd4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76940cf7f0 con 0x7f769415e880 2024-03-29T04:08:00.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f7699dd4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f76940cfe00 con 0x7f76940b4340 2024-03-29T04:08:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f7699dd4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f76940cea70 con 0x7f769415ec60 2024-03-29T04:08:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f76937fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f769415e880 0x7f76940cdd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35692/0 (socket says 172.21.15.97:35692) 2024-03-29T04:08:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f76937fe640 1 -- 172.21.15.97:0/301008817 learned_addr learned my addr 172.21.15.97:0/301008817 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f7693fff640 1 --2- 172.21.15.97:0/301008817 >> v2:172.21.15.154:3300/0 conn(0x7f76940b4340 0x7f76940d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f7692ffd640 1 --2- 172.21.15.97:0/301008817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f769415ec60 0x7f76940ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f76927fc640 1 -- 172.21.15.97:0/301008817 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091219169 0 0) 0x7f76940cf7f0 con 0x7f769415e880 2024-03-29T04:08:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f76927fc640 1 -- 172.21.15.97:0/301008817 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f767c003520 con 0x7f769415e880 2024-03-29T04:08:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f7692ffd640 1 -- 172.21.15.97:0/301008817 >> v2:172.21.15.154:3300/0 conn(0x7f76940b4340 msgr2=0x7f76940d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f7692ffd640 1 --2- 172.21.15.97:0/301008817 >> v2:172.21.15.154:3300/0 conn(0x7f76940b4340 0x7f76940d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f7692ffd640 1 -- 172.21.15.97:0/301008817 >> v1:172.21.15.97:6789/0 conn(0x7f769415e880 legacy=0x7f76940cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f7692ffd640 1 -- 172.21.15.97:0/301008817 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76940ced50 con 0x7f769415ec60 2024-03-29T04:08:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.231+0000 7f7693fff640 1 --2- 172.21.15.97:0/301008817 >> v2:172.21.15.154:3300/0 conn(0x7f76940b4340 0x7f76940d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.232+0000 7f7692ffd640 1 --2- 172.21.15.97:0/301008817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f769415ec60 0x7f76940ce450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f76840097a0 tx=0x7f768402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dc389b0c9fc1268 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.232+0000 7f76927fc640 1 -- 172.21.15.97:0/301008817 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684035020 con 0x7f769415ec60 2024-03-29T04:08:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.232+0000 7f76927fc640 1 -- 172.21.15.97:0/301008817 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7684031830 con 0x7f769415ec60 2024-03-29T04:08:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.232+0000 7f76927fc640 1 -- 172.21.15.97:0/301008817 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684031c00 con 0x7f769415ec60 2024-03-29T04:08:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.233+0000 7f7699dd4640 1 -- 172.21.15.97:0/301008817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f769415ec60 msgr2=0x7f76940ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.233+0000 7f7699dd4640 1 --2- 172.21.15.97:0/301008817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f769415ec60 0x7f76940ce450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f76840097a0 tx=0x7f768402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.233+0000 7f7699dd4640 1 -- 172.21.15.97:0/301008817 shutdown_connections 2024-03-29T04:08:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.233+0000 7f7699dd4640 1 --2- 172.21.15.97:0/301008817 >> v2:172.21.15.154:3300/0 conn(0x7f76940b4340 0x7f76940d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.233+0000 7f7699dd4640 1 --2- 172.21.15.97:0/301008817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f769415ec60 0x7f76940ce450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.233+0000 7f7699dd4640 1 -- 172.21.15.97:0/301008817 >> 172.21.15.97:0/301008817 conn(0x7f76940b6860 msgr2=0x7f76940b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:00.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.233+0000 7f7699dd4640 1 -- 172.21.15.97:0/301008817 shutdown_connections 2024-03-29T04:08:00.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.233+0000 7f7699dd4640 1 -- 172.21.15.97:0/301008817 wait complete. 2024-03-29T04:08:00.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.234+0000 7f7699dd4640 1 Processor -- start 2024-03-29T04:08:00.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.234+0000 7f7699dd4640 1 -- start start 2024-03-29T04:08:00.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7699dd4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76940b4340 0x7f76941f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:00.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7699dd4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f769415e880 0x7f769406b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:00.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7699dd4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f769415ec60 0x7f769406ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:00.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7699dd4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76940d0950 con 0x7f769415ec60 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7699dd4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f76940d07d0 con 0x7f769415e880 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7699dd4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f76940d0ad0 con 0x7f76940b4340 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7693fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f769415ec60 0x7f769406ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7693fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f769415ec60 0x7f769406ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52246/0 (socket says 172.21.15.97:52246) 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7693fff640 1 -- 172.21.15.97:0/3525352435 learned_addr learned my addr 172.21.15.97:0/3525352435 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7692ffd640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f769415e880 0x7f769406b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f76937fe640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76940b4340 0x7f76941f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7693fff640 1 -- 172.21.15.97:0/3525352435 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76940b4340 msgr2=0x7f76941f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7693fff640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76940b4340 0x7f76941f3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7693fff640 1 -- 172.21.15.97:0/3525352435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f769415e880 msgr2=0x7f769406b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7693fff640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f769415e880 0x7f769406b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.235+0000 7f7693fff640 1 -- 172.21.15.97:0/3525352435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769415b760 con 0x7f769415ec60 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.236+0000 7f7693fff640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f769415ec60 0x7f769406ba80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f768c00ef90 tx=0x7f768c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.236+0000 7f767bfff640 1 -- 172.21.15.97:0/3525352435 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c018020 con 0x7f769415ec60 2024-03-29T04:08:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.236+0000 7f767bfff640 1 -- 172.21.15.97:0/3525352435 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768c014480 con 0x7f769415ec60 2024-03-29T04:08:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.236+0000 7f7699dd4640 1 -- 172.21.15.97:0/3525352435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76941f85d0 con 0x7f769415ec60 2024-03-29T04:08:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.236+0000 7f767bfff640 1 -- 172.21.15.97:0/3525352435 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c014870 con 0x7f769415ec60 2024-03-29T04:08:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.236+0000 7f7699dd4640 1 -- 172.21.15.97:0/3525352435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769415bce0 con 0x7f769415ec60 2024-03-29T04:08:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.237+0000 7f7699dd4640 1 -- 172.21.15.97:0/3525352435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7694068640 con 0x7f769415ec60 2024-03-29T04:08:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.237+0000 7f767bfff640 1 -- 172.21.15.97:0/3525352435 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f768c005d10 con 0x7f769415ec60 2024-03-29T04:08:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.238+0000 7f767bfff640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f768006c530 0x7f768006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.238+0000 7f767bfff640 1 -- 172.21.15.97:0/3525352435 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f768c08c320 con 0x7f769415ec60 2024-03-29T04:08:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.238+0000 7f76937fe640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f768006c530 0x7f768006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.238+0000 7f76937fe640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f768006c530 0x7f768006e9f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7688005880 tx=0x7f7688009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.242+0000 7f767bfff640 1 -- 172.21.15.97:0/3525352435 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f768c022030 con 0x7f769415ec60 2024-03-29T04:08:00.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.452+0000 7f7699dd4640 1 -- 172.21.15.97:0/3525352435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f76941f4040 con 0x7f769415ec60 2024-03-29T04:08:00.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.452+0000 7f767bfff640 1 -- 172.21.15.97:0/3525352435 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f768c010040 con 0x7f769415ec60 2024-03-29T04:08:00.433 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T04:08:00.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.454+0000 7f7699dd4640 1 -- 172.21.15.97:0/3525352435 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f768006c530 msgr2=0x7f768006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:00.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.454+0000 7f7699dd4640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f768006c530 0x7f768006e9f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7688005880 tx=0x7f7688009040 comp rx=0 tx=0).stop 2024-03-29T04:08:00.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.454+0000 7f7699dd4640 1 -- 172.21.15.97:0/3525352435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f769415ec60 msgr2=0x7f769406ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:00.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.454+0000 7f7699dd4640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f769415ec60 0x7f769406ba80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f768c00ef90 tx=0x7f768c009f90 comp rx=0 tx=0).stop 2024-03-29T04:08:00.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.454+0000 7f7699dd4640 1 -- 172.21.15.97:0/3525352435 shutdown_connections 2024-03-29T04:08:00.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.454+0000 7f7699dd4640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f768006c530 0x7f768006e9f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.454+0000 7f7699dd4640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f769415ec60 0x7f769406ba80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.454+0000 7f7699dd4640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f769415e880 0x7f769406b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.454+0000 7f7699dd4640 1 --2- 172.21.15.97:0/3525352435 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f76940b4340 0x7f76941f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.454+0000 7f7699dd4640 1 -- 172.21.15.97:0/3525352435 >> 172.21.15.97:0/3525352435 conn(0x7f76940b6860 msgr2=0x7f76940d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.454+0000 7f7699dd4640 1 -- 172.21.15.97:0/3525352435 shutdown_connections 2024-03-29T04:08:00.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:00.454+0000 7f7699dd4640 1 -- 172.21.15.97:0/3525352435 wait complete. 2024-03-29T04:08:00.445 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:08:00.446 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:08:00.446 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-03-29T04:08:00.446 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-29T04:08:00.446 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-29T04:08:00.446 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:08:00.446 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:08:00.446 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:08:00.446 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-29T04:08:00.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:00.446 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-03-29T04:08:00.462 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:08:00.462 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:08:00.462 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:08:00.462 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:00.462 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-03-29T04:08:00.462 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:00.462 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:08:00.462 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:00.463 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-03-29T04:08:00.463 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:08:00.463 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:00.463 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-03-29T04:08:00.463 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:08:00.463 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:00.463 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:08:00.463 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:00.463 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:08:00.463 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42704sec preferred_lft 42704sec 2024-03-29T04:08:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2024-03-29T04:08:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:00.465 INFO:teuthology.orchestra.run.smithi154.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:08:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether d6:7d:69:db:81:04 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:08:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::60a9:efff:fe1a:efc0/64 scope link 2024-03-29T04:08:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:00.465 INFO:teuthology.orchestra.run.smithi154.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:08:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether d6:7d:69:db:81:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:08:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::d47d:69ff:fedb:8104/64 scope link tentative 2024-03-29T04:08:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:00.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:00.466 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:08:00.528 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-03-29T04:08:00.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:00.529 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-03-29T04:08:00.585 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-03-29T04:08:00.585 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:08:00.585 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:08:00.585 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:08:00.585 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:08:00.586 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:08:00.586 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-29T04:08:00.586 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:08:00.586 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:08:00.586 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:08:00.586 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-29T04:08:00.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:08:00.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:00.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:00.720 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-29T04:08:00.735 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T04:08:00.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:00.736 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list-id 2024-03-29T04:08:00.791 INFO:teuthology.orchestra.run.smithi154.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-29T04:08:00.791 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:08:00.791 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-29T04:08:00.791 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:08:00.791 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-29T04:08:00.791 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-29T04:08:00.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:08:00.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:00.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:00.916 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-29T04:08:00.945 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:08:00.945 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:08:00.946 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:08:00.946 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:00.946 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-03-29T04:08:00.946 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:00.946 INFO:teuthology.orchestra.run.smithi154.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:08:00.946 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether b2:8c:1b:56:e9:ba brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:08:00.946 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:08:00.946 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:00.946 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::b08c:1bff:fe56:e9ba/64 scope link tentative 2024-03-29T04:08:00.946 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:00.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:00.947 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-29T04:08:01.014 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:08:01.015 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:08:01.015 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:08:01.015 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:01.015 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-03-29T04:08:01.015 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:01.015 INFO:teuthology.orchestra.run.smithi154.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:08:01.015 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether b2:8c:1b:56:e9:ba brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:08:01.015 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:08:01.015 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:01.015 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::b08c:1bff:fe56:e9ba/64 scope link tentative 2024-03-29T04:08:01.015 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:01.016 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-29T04:08:01.016 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:08:01.017 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-29T04:08:01.017 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-29T04:08:01.017 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-29T04:08:01.017 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-29T04:08:01.017 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-29T04:08:01.017 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-29T04:08:01.017 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-29T04:08:01.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:08:01.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:01.184 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:08:01.184 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-29T04:08:01.184 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.1 up 2024-03-29T04:08:01.184 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set dev brx.1 master ceph-brx 2024-03-29T04:08:01.184 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-29T04:08:01.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:08:01.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:01.267 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:08:01.267 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-29T04:08:01.282 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-29T04:08:01.282 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:08:01.282 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-29T04:08:01.337 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-03-29T04:08:01.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:01.338 DEBUG:teuthology.orchestra.run.smithi154:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:08:01.422 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:08:01.422 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:08:01.429 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:08:01.429 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:08:01.429 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.154:3300,172.21.15.154:3301" 2024-03-29T04:08:01.429 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: 3@51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f.cephfs=/ 2024-03-29T04:08:01.430 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.97:3300/172.21.15.154:3300/172.21.15.154:3301" will pass to kernel 2024-03-29T04:08:01.430 INFO:teuthology.orchestra.run.smithi154.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-29T04:08:01.430 INFO:teuthology.orchestra.run.smithi154.stdout: You just mounted a file system that supports labels which does not 2024-03-29T04:08:01.430 INFO:teuthology.orchestra.run.smithi154.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-29T04:08:01.430 INFO:teuthology.orchestra.run.smithi154.stdout: applications will generate AVC messages and not be allowed access to 2024-03-29T04:08:01.430 INFO:teuthology.orchestra.run.smithi154.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-29T04:08:01.433 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-29T04:08:01.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:01.433 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-29T04:08:01.465 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:08:01.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:08:01.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:01.562 INFO:teuthology.orchestra.run.smithi154.stdout:client4541 2024-03-29T04:08:01.562 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:08:01.562 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:01.578 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:08:01.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:08:01.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:01.672 INFO:teuthology.orchestra.run.smithi154.stdout:client4541 2024-03-29T04:08:01.673 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:08:01.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:08:01.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:01.727 INFO:teuthology.orchestra.run.smithi154.stdout:client4541 2024-03-29T04:08:01.728 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f.client4541/status') 2024-03-29T04:08:01.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:08:01.788 INFO:teuthology.orchestra.run.smithi154.stdout:instance: client.4541 (3)192.168.0.2:0/311623388 2024-03-29T04:08:01.789 INFO:teuthology.orchestra.run.smithi154.stdout:blocklisted: false 2024-03-29T04:08:01.789 INFO:teuthology.orchestra.run.smithi154.stderr:0+1 records in 2024-03-29T04:08:01.789 INFO:teuthology.orchestra.run.smithi154.stderr:0+1 records out 2024-03-29T04:08:01.789 INFO:teuthology.orchestra.run.smithi154.stderr:68 bytes copied, 2.294e-05 s, 3.0 MB/s 2024-03-29T04:08:01.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:01.796 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-29T04:08:02.082 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:02.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.213+0000 7f5daebe8640 1 Processor -- start 2024-03-29T04:08:02.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.213+0000 7f5daebe8640 1 -- start start 2024-03-29T04:08:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5daebe8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da815e6e0 0x7f5da809b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5daebe8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5da80d0660 0x7f5da81623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5daebe8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da80d1a10 con 0x7f5da81628e0 2024-03-29T04:08:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5daebe8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f5da80d0e90 con 0x7f5da80d0660 2024-03-29T04:08:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5daebe8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5da80d1170 con 0x7f5da815e6e0 2024-03-29T04:08:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5dad185640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5da81628e0 0x7f5da80b30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33408/0 (socket says 172.21.15.97:33408) 2024-03-29T04:08:02.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5dad185640 1 -- 172.21.15.97:0/2180664500 learned_addr learned my addr 172.21.15.97:0/2180664500 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5dac984640 1 --2- 172.21.15.97:0/2180664500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da815e6e0 0x7f5da809b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5d9ffff640 1 --2- 172.21.15.97:0/2180664500 >> v2:172.21.15.154:3300/0 conn(0x7f5da80d0660 0x7f5da81623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5d9f7fe640 1 -- 172.21.15.97:0/2180664500 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1881387654 0 0) 0x7f5da80d1a10 con 0x7f5da81628e0 2024-03-29T04:08:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5d9f7fe640 1 -- 172.21.15.97:0/2180664500 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d90003520 con 0x7f5da81628e0 2024-03-29T04:08:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5d9ffff640 1 -- 172.21.15.97:0/2180664500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da815e6e0 msgr2=0x7f5da809b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5d9ffff640 1 --2- 172.21.15.97:0/2180664500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da815e6e0 0x7f5da809b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5d9ffff640 1 -- 172.21.15.97:0/2180664500 >> v1:172.21.15.97:6789/0 conn(0x7f5da81628e0 legacy=0x7f5da80b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.214+0000 7f5d9ffff640 1 -- 172.21.15.97:0/2180664500 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da80d1450 con 0x7f5da80d0660 2024-03-29T04:08:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.215+0000 7f5d9ffff640 1 --2- 172.21.15.97:0/2180664500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da80d0660 0x7f5da81623a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d980097d0 tx=0x7f5d9802b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d457c6754bf0e8c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.215+0000 7f5d9f7fe640 1 -- 172.21.15.97:0/2180664500 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98035020 con 0x7f5da80d0660 2024-03-29T04:08:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.215+0000 7f5d9f7fe640 1 -- 172.21.15.97:0/2180664500 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d98031800 con 0x7f5da80d0660 2024-03-29T04:08:02.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.215+0000 7f5d9f7fe640 1 -- 172.21.15.97:0/2180664500 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98031bb0 con 0x7f5da80d0660 2024-03-29T04:08:02.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.216+0000 7f5daebe8640 1 -- 172.21.15.97:0/2180664500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da80d0660 msgr2=0x7f5da81623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.216+0000 7f5daebe8640 1 --2- 172.21.15.97:0/2180664500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da80d0660 0x7f5da81623a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d980097d0 tx=0x7f5d9802b8a0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.216+0000 7f5daebe8640 1 -- 172.21.15.97:0/2180664500 shutdown_connections 2024-03-29T04:08:02.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.216+0000 7f5daebe8640 1 --2- 172.21.15.97:0/2180664500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da80d0660 0x7f5da81623a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.216+0000 7f5daebe8640 1 --2- 172.21.15.97:0/2180664500 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da815e6e0 0x7f5da809b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.216+0000 7f5daebe8640 1 -- 172.21.15.97:0/2180664500 >> 172.21.15.97:0/2180664500 conn(0x7f5da80b3f10 msgr2=0x7f5da80b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:02.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.216+0000 7f5daebe8640 1 -- 172.21.15.97:0/2180664500 shutdown_connections 2024-03-29T04:08:02.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.216+0000 7f5daebe8640 1 -- 172.21.15.97:0/2180664500 wait complete. 2024-03-29T04:08:02.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.217+0000 7f5daebe8640 1 Processor -- start 2024-03-29T04:08:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.217+0000 7f5daebe8640 1 -- start start 2024-03-29T04:08:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.218+0000 7f5daebe8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da80d0660 0x7f5da81f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.218+0000 7f5daebe8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da815e6e0 0x7f5da80b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.218+0000 7f5daebe8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da81628e0 0x7f5da80afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.218+0000 7f5daebe8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5da806d910 con 0x7f5da80d0660 2024-03-29T04:08:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.218+0000 7f5daebe8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5da80b1d80 con 0x7f5da815e6e0 2024-03-29T04:08:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.218+0000 7f5daebe8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5da815ee30 con 0x7f5da81628e0 2024-03-29T04:08:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.218+0000 7f5dac984640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da80d0660 0x7f5da81f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.218+0000 7f5d9ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da815e6e0 0x7f5da80b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.218+0000 7f5dac984640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da80d0660 0x7f5da81f7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39354/0 (socket says 172.21.15.97:39354) 2024-03-29T04:08:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.218+0000 7f5dac984640 1 -- 172.21.15.97:0/3842675109 learned_addr learned my addr 172.21.15.97:0/3842675109 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.218+0000 7f5dad185640 1 --2- 172.21.15.97:0/3842675109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da81628e0 0x7f5da80afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.219+0000 7f5dac984640 1 -- 172.21.15.97:0/3842675109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da81628e0 msgr2=0x7f5da80afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.219+0000 7f5dac984640 1 --2- 172.21.15.97:0/3842675109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da81628e0 0x7f5da80afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.219+0000 7f5dac984640 1 -- 172.21.15.97:0/3842675109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da815e6e0 msgr2=0x7f5da80b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.219+0000 7f5dac984640 1 --2- 172.21.15.97:0/3842675109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da815e6e0 0x7f5da80b6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.219+0000 7f5dac984640 1 -- 172.21.15.97:0/3842675109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da815f110 con 0x7f5da80d0660 2024-03-29T04:08:02.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.219+0000 7f5dac984640 1 --2- 172.21.15.97:0/3842675109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da80d0660 0x7f5da81f7560 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009db0 tx=0x7f5da400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:02.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.219+0000 7f5d9d7fa640 1 -- 172.21.15.97:0/3842675109 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4018020 con 0x7f5da80d0660 2024-03-29T04:08:02.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.219+0000 7f5daebe8640 1 -- 172.21.15.97:0/3842675109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da815f3f0 con 0x7f5da80d0660 2024-03-29T04:08:02.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.219+0000 7f5d9d7fa640 1 -- 172.21.15.97:0/3842675109 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5da4014480 con 0x7f5da80d0660 2024-03-29T04:08:02.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.219+0000 7f5d9d7fa640 1 -- 172.21.15.97:0/3842675109 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4014850 con 0x7f5da80d0660 2024-03-29T04:08:02.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.219+0000 7f5daebe8640 1 -- 172.21.15.97:0/3842675109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da815f9b0 con 0x7f5da80d0660 2024-03-29T04:08:02.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.220+0000 7f5daebe8640 1 -- 172.21.15.97:0/3842675109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da8068640 con 0x7f5da80d0660 2024-03-29T04:08:02.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.220+0000 7f5d9d7fa640 1 -- 172.21.15.97:0/3842675109 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5da4005d10 con 0x7f5da80d0660 2024-03-29T04:08:02.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.221+0000 7f5d9d7fa640 1 --2- 172.21.15.97:0/3842675109 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5d9406c530 0x7f5d9406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.221+0000 7f5d9d7fa640 1 -- 172.21.15.97:0/3842675109 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5da408c180 con 0x7f5da80d0660 2024-03-29T04:08:02.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.221+0000 7f5d9ffff640 1 --2- 172.21.15.97:0/3842675109 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5d9406c530 0x7f5d9406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.221+0000 7f5d9ffff640 1 --2- 172.21.15.97:0/3842675109 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5d9406c530 0x7f5d9406e9f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d98036000 tx=0x7f5d98008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:02.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.225+0000 7f5d9d7fa640 1 -- 172.21.15.97:0/3842675109 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5da4022070 con 0x7f5da80d0660 2024-03-29T04:08:02.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.400+0000 7f5daebe8640 1 -- 172.21.15.97:0/3842675109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5da80b4b00 con 0x7f5da80d0660 2024-03-29T04:08:02.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.401+0000 7f5d9d7fa640 1 -- 172.21.15.97:0/3842675109 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f5da4010040 con 0x7f5da80d0660 2024-03-29T04:08:02.382 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:02.382 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.403+0000 7f5daebe8640 1 -- 172.21.15.97:0/3842675109 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5d9406c530 msgr2=0x7f5d9406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.403+0000 7f5daebe8640 1 --2- 172.21.15.97:0/3842675109 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5d9406c530 0x7f5d9406e9f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d98036000 tx=0x7f5d98008030 comp rx=0 tx=0).stop 2024-03-29T04:08:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.403+0000 7f5daebe8640 1 -- 172.21.15.97:0/3842675109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da80d0660 msgr2=0x7f5da81f7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.403+0000 7f5daebe8640 1 --2- 172.21.15.97:0/3842675109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da80d0660 0x7f5da81f7560 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009db0 tx=0x7f5da400c490 comp rx=0 tx=0).stop 2024-03-29T04:08:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.403+0000 7f5daebe8640 1 -- 172.21.15.97:0/3842675109 shutdown_connections 2024-03-29T04:08:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.403+0000 7f5daebe8640 1 --2- 172.21.15.97:0/3842675109 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5d9406c530 0x7f5d9406e9f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.403+0000 7f5daebe8640 1 --2- 172.21.15.97:0/3842675109 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5da81628e0 0x7f5da80afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.403+0000 7f5daebe8640 1 --2- 172.21.15.97:0/3842675109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5da815e6e0 0x7f5da80b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.403+0000 7f5daebe8640 1 --2- 172.21.15.97:0/3842675109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da80d0660 0x7f5da81f7560 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.403+0000 7f5daebe8640 1 -- 172.21.15.97:0/3842675109 >> 172.21.15.97:0/3842675109 conn(0x7f5da80b3f10 msgr2=0x7f5da80b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.403+0000 7f5daebe8640 1 -- 172.21.15.97:0/3842675109 shutdown_connections 2024-03-29T04:08:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.403+0000 7f5daebe8640 1 -- 172.21.15.97:0/3842675109 wait complete. 2024-03-29T04:08:02.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:07:38.618704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:07:39.724339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:08:02.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:08:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.518+0000 7f91a3835640 1 Processor -- start 2024-03-29T04:08:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.518+0000 7f91a3835640 1 -- start start 2024-03-29T04:08:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a3835640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f919c15e8b0 0x7f919c0b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a3835640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f919c0b6020 0x7f919c0b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a3835640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f919c0b6500 con 0x7f919c06b510 2024-03-29T04:08:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a3835640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f919c0b67e0 con 0x7f919c15e8b0 2024-03-29T04:08:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a3835640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f919c0b4470 con 0x7f919c0b6020 2024-03-29T04:08:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a15d1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f919c15e8b0 0x7f919c0b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a15d1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f919c15e8b0 0x7f919c0b2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39364/0 (socket says 172.21.15.97:39364) 2024-03-29T04:08:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a15d1640 1 -- 172.21.15.97:0/3048007856 learned_addr learned my addr 172.21.15.97:0/3048007856 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a0dd0640 1 --2- 172.21.15.97:0/3048007856 >> v2:172.21.15.154:3300/0 conn(0x7f919c0b6020 0x7f919c0b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a15d1640 1 -- 172.21.15.97:0/3048007856 >> v2:172.21.15.154:3300/0 conn(0x7f919c0b6020 msgr2=0x7f919c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:02.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a15d1640 1 --2- 172.21.15.97:0/3048007856 >> v2:172.21.15.154:3300/0 conn(0x7f919c0b6020 0x7f919c0b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a15d1640 1 -- 172.21.15.97:0/3048007856 >> v1:172.21.15.154:6789/0 conn(0x7f919c06b510 legacy=0x7f919c0ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a15d1640 1 -- 172.21.15.97:0/3048007856 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c0b4750 con 0x7f919c15e8b0 2024-03-29T04:08:02.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.520+0000 7f91a15d1640 1 --2- 172.21.15.97:0/3048007856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919c15e8b0 0x7f919c0b2a40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f919c071370 tx=0x7f919802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9605ed3e664b09f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:02.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.521+0000 7f9193fff640 1 -- 172.21.15.97:0/3048007856 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198035020 con 0x7f919c15e8b0 2024-03-29T04:08:02.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.521+0000 7f9193fff640 1 -- 172.21.15.97:0/3048007856 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9198031870 con 0x7f919c15e8b0 2024-03-29T04:08:02.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.521+0000 7f9193fff640 1 -- 172.21.15.97:0/3048007856 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198031c40 con 0x7f919c15e8b0 2024-03-29T04:08:02.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.521+0000 7f91a3835640 1 -- 172.21.15.97:0/3048007856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919c15e8b0 msgr2=0x7f919c0b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.521+0000 7f91a3835640 1 --2- 172.21.15.97:0/3048007856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919c15e8b0 0x7f919c0b2a40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f919c071370 tx=0x7f919802b910 comp rx=0 tx=0).stop 2024-03-29T04:08:02.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.521+0000 7f91a3835640 1 -- 172.21.15.97:0/3048007856 shutdown_connections 2024-03-29T04:08:02.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.521+0000 7f91a3835640 1 --2- 172.21.15.97:0/3048007856 >> v2:172.21.15.154:3300/0 conn(0x7f919c0b6020 0x7f919c0b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.521+0000 7f91a3835640 1 --2- 172.21.15.97:0/3048007856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919c15e8b0 0x7f919c0b2a40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.521+0000 7f91a3835640 1 -- 172.21.15.97:0/3048007856 >> 172.21.15.97:0/3048007856 conn(0x7f919c072e40 msgr2=0x7f919c0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.521+0000 7f91a3835640 1 -- 172.21.15.97:0/3048007856 shutdown_connections 2024-03-29T04:08:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.521+0000 7f91a3835640 1 -- 172.21.15.97:0/3048007856 wait complete. 2024-03-29T04:08:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.522+0000 7f91a3835640 1 Processor -- start 2024-03-29T04:08:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.522+0000 7f91a3835640 1 -- start start 2024-03-29T04:08:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a3835640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919c06b510 0x7f919c153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a3835640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f919c0b6020 0x7f919c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a3835640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f919c15e8b0 0x7f919c152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a3835640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f919c0b2680 con 0x7f919c06b510 2024-03-29T04:08:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a3835640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f919c0b4b30 con 0x7f919c15e8b0 2024-03-29T04:08:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a3835640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f919c1f4d50 con 0x7f919c0b6020 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a15d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919c06b510 0x7f919c153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a15d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919c06b510 0x7f919c153780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39376/0 (socket says 172.21.15.97:39376) 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a15d1640 1 -- 172.21.15.97:0/3381560467 learned_addr learned my addr 172.21.15.97:0/3381560467 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a0dd0640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f919c0b6020 0x7f919c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a1dd2640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f919c15e8b0 0x7f919c152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a15d1640 1 -- 172.21.15.97:0/3381560467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f919c0b6020 msgr2=0x7f919c157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a15d1640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f919c0b6020 0x7f919c157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a15d1640 1 -- 172.21.15.97:0/3381560467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f919c15e8b0 msgr2=0x7f919c152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a15d1640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f919c15e8b0 0x7f919c152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a15d1640 1 -- 172.21.15.97:0/3381560467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c157870 con 0x7f919c06b510 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.523+0000 7f91a1dd2640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f919c15e8b0 0x7f919c152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.524+0000 7f91a15d1640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919c06b510 0x7f919c153780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f919802b950 tx=0x7f91980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.524+0000 7f9191ffb640 1 -- 172.21.15.97:0/3381560467 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198040070 con 0x7f919c06b510 2024-03-29T04:08:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.524+0000 7f9191ffb640 1 -- 172.21.15.97:0/3381560467 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9198030d00 con 0x7f919c06b510 2024-03-29T04:08:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.524+0000 7f9191ffb640 1 -- 172.21.15.97:0/3381560467 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198035040 con 0x7f919c06b510 2024-03-29T04:08:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.524+0000 7f91a3835640 1 -- 172.21.15.97:0/3381560467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919c153e10 con 0x7f919c06b510 2024-03-29T04:08:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.524+0000 7f91a3835640 1 -- 172.21.15.97:0/3381560467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919c154cc0 con 0x7f919c06b510 2024-03-29T04:08:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.525+0000 7f91a3835640 1 -- 172.21.15.97:0/3381560467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f919c09f6a0 con 0x7f919c06b510 2024-03-29T04:08:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.525+0000 7f9191ffb640 1 -- 172.21.15.97:0/3381560467 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f919804b060 con 0x7f919c06b510 2024-03-29T04:08:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.526+0000 7f9191ffb640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f918006c530 0x7f918006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.526+0000 7f91a0dd0640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f918006c530 0x7f918006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.526+0000 7f9191ffb640 1 -- 172.21.15.97:0/3381560467 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f91980ad310 con 0x7f919c06b510 2024-03-29T04:08:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.526+0000 7f91a0dd0640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f918006c530 0x7f918006e9f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f919c15b360 tx=0x7f918c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.530+0000 7f9191ffb640 1 -- 172.21.15.97:0/3381560467 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f919807b9c0 con 0x7f919c06b510 2024-03-29T04:08:02.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.706+0000 7f91a3835640 1 -- 172.21.15.97:0/3381560467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f919c161d00 con 0x7f919c06b510 2024-03-29T04:08:02.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.706+0000 7f9191ffb640 1 -- 172.21.15.97:0/3381560467 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f919807b2c0 con 0x7f919c06b510 2024-03-29T04:08:02.687 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:08:02.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.708+0000 7f91a3835640 1 -- 172.21.15.97:0/3381560467 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f918006c530 msgr2=0x7f918006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.708+0000 7f91a3835640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f918006c530 0x7f918006e9f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f919c15b360 tx=0x7f918c008040 comp rx=0 tx=0).stop 2024-03-29T04:08:02.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.708+0000 7f91a3835640 1 -- 172.21.15.97:0/3381560467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919c06b510 msgr2=0x7f919c153780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.708+0000 7f91a3835640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919c06b510 0x7f919c153780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f919802b950 tx=0x7f91980046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.708+0000 7f91a3835640 1 -- 172.21.15.97:0/3381560467 shutdown_connections 2024-03-29T04:08:02.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.708+0000 7f91a3835640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f918006c530 0x7f918006e9f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.708+0000 7f91a3835640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f919c15e8b0 0x7f919c152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.708+0000 7f91a3835640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f919c0b6020 0x7f919c157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.708+0000 7f91a3835640 1 --2- 172.21.15.97:0/3381560467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919c06b510 0x7f919c153780 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.708+0000 7f91a3835640 1 -- 172.21.15.97:0/3381560467 >> 172.21.15.97:0/3381560467 conn(0x7f919c072e40 msgr2=0x7f919c0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:02.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.708+0000 7f91a3835640 1 -- 172.21.15.97:0/3381560467 shutdown_connections 2024-03-29T04:08:02.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.708+0000 7f91a3835640 1 -- 172.21.15.97:0/3381560467 wait complete. 2024-03-29T04:08:02.700 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:08:02.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.823+0000 7fc3b9a60640 1 Processor -- start 2024-03-29T04:08:02.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.823+0000 7fc3b9a60640 1 -- start start 2024-03-29T04:08:02.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3b9a60640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b40cfde0 0x7fc3b40d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3b9a60640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc3b40d0f90 0x7fc3b40d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3b9a60640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b415ed00 con 0x7fc3b415e920 2024-03-29T04:08:02.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3b9a60640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b40d21e0 con 0x7fc3b40d0f90 2024-03-29T04:08:02.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3b9a60640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b40d2480 con 0x7fc3b40cfde0 2024-03-29T04:08:02.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3b2ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc3b415e920 0x7fc3b40d3240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33424/0 (socket says 172.21.15.97:33424) 2024-03-29T04:08:02.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3b2ffd640 1 -- 172.21.15.97:0/434699647 learned_addr learned my addr 172.21.15.97:0/434699647 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:02.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3aa7fc640 1 --2- 172.21.15.97:0/434699647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b40cfde0 0x7fc3b40d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3b37fe640 1 --2- 172.21.15.97:0/434699647 >> v2:172.21.15.154:3300/0 conn(0x7fc3b40d0f90 0x7fc3b40d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3b27fc640 1 -- 172.21.15.97:0/434699647 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3932846650 0 0) 0x7fc3b415ed00 con 0x7fc3b415e920 2024-03-29T04:08:02.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3b27fc640 1 -- 172.21.15.97:0/434699647 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3a0003520 con 0x7fc3b415e920 2024-03-29T04:08:02.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3aa7fc640 1 -- 172.21.15.97:0/434699647 >> v2:172.21.15.154:3300/0 conn(0x7fc3b40d0f90 msgr2=0x7fc3b40d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:02.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3aa7fc640 1 --2- 172.21.15.97:0/434699647 >> v2:172.21.15.154:3300/0 conn(0x7fc3b40d0f90 0x7fc3b40d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3aa7fc640 1 -- 172.21.15.97:0/434699647 >> v1:172.21.15.97:6789/0 conn(0x7fc3b415e920 legacy=0x7fc3b40d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.824+0000 7fc3aa7fc640 1 -- 172.21.15.97:0/434699647 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b40d2760 con 0x7fc3b40cfde0 2024-03-29T04:08:02.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.825+0000 7fc3aa7fc640 1 --2- 172.21.15.97:0/434699647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b40cfde0 0x7fc3b40d0240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0097d0 tx=0x7fc39c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=929d7645750c2286 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.825+0000 7fc3b27fc640 1 -- 172.21.15.97:0/434699647 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c035020 con 0x7fc3b40cfde0 2024-03-29T04:08:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.825+0000 7fc3b27fc640 1 -- 172.21.15.97:0/434699647 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc39c031800 con 0x7fc3b40cfde0 2024-03-29T04:08:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.825+0000 7fc3b27fc640 1 -- 172.21.15.97:0/434699647 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c031bd0 con 0x7fc3b40cfde0 2024-03-29T04:08:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.826+0000 7fc3b9a60640 1 -- 172.21.15.97:0/434699647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b40cfde0 msgr2=0x7fc3b40d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.826+0000 7fc3b9a60640 1 --2- 172.21.15.97:0/434699647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b40cfde0 0x7fc3b40d0240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0097d0 tx=0x7fc39c02b8a0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.826+0000 7fc3b9a60640 1 -- 172.21.15.97:0/434699647 shutdown_connections 2024-03-29T04:08:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.826+0000 7fc3b9a60640 1 --2- 172.21.15.97:0/434699647 >> v2:172.21.15.154:3300/0 conn(0x7fc3b40d0f90 0x7fc3b40d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.826+0000 7fc3b9a60640 1 --2- 172.21.15.97:0/434699647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b40cfde0 0x7fc3b40d0240 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.826+0000 7fc3b9a60640 1 -- 172.21.15.97:0/434699647 >> 172.21.15.97:0/434699647 conn(0x7fc3b408bfc0 msgr2=0x7fc3b40d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.826+0000 7fc3b9a60640 1 -- 172.21.15.97:0/434699647 shutdown_connections 2024-03-29T04:08:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.826+0000 7fc3b9a60640 1 -- 172.21.15.97:0/434699647 wait complete. 2024-03-29T04:08:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.827+0000 7fc3b9a60640 1 Processor -- start 2024-03-29T04:08:02.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.827+0000 7fc3b9a60640 1 -- start start 2024-03-29T04:08:02.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b9a60640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b40cfde0 0x7fc3b41f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b9a60640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b40d0f90 0x7fc3b41f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b9a60640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b415e920 0x7fc3b415f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b9a60640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b40d4ef0 con 0x7fc3b415e920 2024-03-29T04:08:02.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b9a60640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b40d2d20 con 0x7fc3b40cfde0 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b9a60640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b40d5070 con 0x7fc3b40d0f90 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b415e920 0x7fc3b415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3aa7fc640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b40d0f90 0x7fc3b41f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b415e920 0x7fc3b415f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39384/0 (socket says 172.21.15.97:39384) 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b40cfde0 0x7fc3b41f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b37fe640 1 -- 172.21.15.97:0/2686561527 learned_addr learned my addr 172.21.15.97:0/2686561527 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b37fe640 1 -- 172.21.15.97:0/2686561527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b40d0f90 msgr2=0x7fc3b41f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b37fe640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b40d0f90 0x7fc3b41f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b37fe640 1 -- 172.21.15.97:0/2686561527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b40cfde0 msgr2=0x7fc3b41f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b37fe640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b40cfde0 0x7fc3b41f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b37fe640 1 -- 172.21.15.97:0/2686561527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b41f54c0 con 0x7fc3b415e920 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3aa7fc640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b40d0f90 0x7fc3b41f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b2ffd640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b40cfde0 0x7fc3b41f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.828+0000 7fc3b37fe640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b415e920 0x7fc3b415f450 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00efc0 tx=0x7fc3ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.829+0000 7fc3b0ff9640 1 -- 172.21.15.97:0/2686561527 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac018020 con 0x7fc3b415e920 2024-03-29T04:08:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.829+0000 7fc3b0ff9640 1 -- 172.21.15.97:0/2686561527 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ac014480 con 0x7fc3b415e920 2024-03-29T04:08:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.829+0000 7fc3b9a60640 1 -- 172.21.15.97:0/2686561527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b41f57a0 con 0x7fc3b415e920 2024-03-29T04:08:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.829+0000 7fc3b0ff9640 1 -- 172.21.15.97:0/2686561527 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac014870 con 0x7fc3b415e920 2024-03-29T04:08:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.829+0000 7fc3b9a60640 1 -- 172.21.15.97:0/2686561527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b40c93e0 con 0x7fc3b415e920 2024-03-29T04:08:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.830+0000 7fc3b9a60640 1 -- 172.21.15.97:0/2686561527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b40a2560 con 0x7fc3b415e920 2024-03-29T04:08:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.830+0000 7fc3b0ff9640 1 -- 172.21.15.97:0/2686561527 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc3ac005d10 con 0x7fc3b415e920 2024-03-29T04:08:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.830+0000 7fc3b0ff9640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc39406c560 0x7fc39406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.831+0000 7fc3b0ff9640 1 -- 172.21.15.97:0/2686561527 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc3ac08c0b0 con 0x7fc3b415e920 2024-03-29T04:08:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.831+0000 7fc3b2ffd640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc39406c560 0x7fc39406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.831+0000 7fc3b2ffd640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc39406c560 0x7fc39406ea20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4005880 tx=0x7fc3a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:02.835+0000 7fc3b0ff9640 1 -- 172.21.15.97:0/2686561527 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc3ac022070 con 0x7fc3b415e920 2024-03-29T04:08:02.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.008+0000 7fc3b9a60640 1 -- 172.21.15.97:0/2686561527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc3b404f9d0 con 0x7fc3b415e920 2024-03-29T04:08:02.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.008+0000 7fc3b0ff9640 1 -- 172.21.15.97:0/2686561527 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc3ac010040 con 0x7fc3b415e920 2024-03-29T04:08:02.988 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:08:02.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.010+0000 7fc3b9a60640 1 -- 172.21.15.97:0/2686561527 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc39406c560 msgr2=0x7fc39406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.010+0000 7fc3b9a60640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc39406c560 0x7fc39406ea20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4005880 tx=0x7fc3a4009040 comp rx=0 tx=0).stop 2024-03-29T04:08:02.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.010+0000 7fc3b9a60640 1 -- 172.21.15.97:0/2686561527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b415e920 msgr2=0x7fc3b415f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:02.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.010+0000 7fc3b9a60640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b415e920 0x7fc3b415f450 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00efc0 tx=0x7fc3ac00c490 comp rx=0 tx=0).stop 2024-03-29T04:08:02.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.010+0000 7fc3b9a60640 1 -- 172.21.15.97:0/2686561527 shutdown_connections 2024-03-29T04:08:02.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.010+0000 7fc3b9a60640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc39406c560 0x7fc39406ea20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.010+0000 7fc3b9a60640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b415e920 0x7fc3b415f450 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.010+0000 7fc3b9a60640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc3b40d0f90 0x7fc3b41f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.010+0000 7fc3b9a60640 1 --2- 172.21.15.97:0/2686561527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3b40cfde0 0x7fc3b41f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:02.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.011+0000 7fc3b9a60640 1 -- 172.21.15.97:0/2686561527 >> 172.21.15.97:0/2686561527 conn(0x7fc3b408bfc0 msgr2=0x7fc3b40b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:02.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.011+0000 7fc3b9a60640 1 -- 172.21.15.97:0/2686561527 shutdown_connections 2024-03-29T04:08:02.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.011+0000 7fc3b9a60640 1 -- 172.21.15.97:0/2686561527 wait complete. 2024-03-29T04:08:03.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:08:03.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.124+0000 7f3202782640 1 Processor -- start 2024-03-29T04:08:03.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.124+0000 7f3202782640 1 -- start start 2024-03-29T04:08:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f3202782640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f31fc0d0a90 0x7f31fc0d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f3202782640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f31fc09ba40 0x7f31fc15cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f3202782640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31fc0d13b0 con 0x7f31fc0d0080 2024-03-29T04:08:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f3202782640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f31fc0cd410 con 0x7f31fc09ba40 2024-03-29T04:08:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f3202782640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f31fc0cd6f0 con 0x7f31fc0d0a90 2024-03-29T04:08:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f3200d1f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f31fc09ba40 0x7f31fc15cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f31fb7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f31fc0d0080 0x7f31fc15c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33440/0 (socket says 172.21.15.97:33440) 2024-03-29T04:08:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f31fb7fe640 1 -- 172.21.15.97:0/1581108958 learned_addr learned my addr 172.21.15.97:0/1581108958 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f31fbfff640 1 --2- 172.21.15.97:0/1581108958 >> v2:172.21.15.154:3300/0 conn(0x7f31fc0d0a90 0x7f31fc0d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f3200d1f640 1 -- 172.21.15.97:0/1581108958 >> v2:172.21.15.154:3300/0 conn(0x7f31fc0d0a90 msgr2=0x7f31fc0d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f3200d1f640 1 --2- 172.21.15.97:0/1581108958 >> v2:172.21.15.154:3300/0 conn(0x7f31fc0d0a90 0x7f31fc0d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f3200d1f640 1 -- 172.21.15.97:0/1581108958 >> v1:172.21.15.97:6789/0 conn(0x7f31fc0d0080 legacy=0x7f31fc15c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f3200d1f640 1 -- 172.21.15.97:0/1581108958 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc0cd9d0 con 0x7f31fc09ba40 2024-03-29T04:08:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.125+0000 7f31faffd640 1 -- 172.21.15.97:0/1581108958 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1994691609 0 0) 0x7f31fc0d13b0 con 0x7f31fc0d0080 2024-03-29T04:08:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.126+0000 7f3200d1f640 1 --2- 172.21.15.97:0/1581108958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31fc09ba40 0x7f31fc15cfd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f31fc072a80 tx=0x7f31f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bde8b8aefdff75 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.126+0000 7f31faffd640 1 -- 172.21.15.97:0/1581108958 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4039070 con 0x7f31fc09ba40 2024-03-29T04:08:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.126+0000 7f31faffd640 1 -- 172.21.15.97:0/1581108958 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f402cc80 con 0x7f31fc09ba40 2024-03-29T04:08:03.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.126+0000 7f31faffd640 1 -- 172.21.15.97:0/1581108958 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4034660 con 0x7f31fc09ba40 2024-03-29T04:08:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.126+0000 7f3202782640 1 -- 172.21.15.97:0/1581108958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31fc09ba40 msgr2=0x7f31fc15cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.126+0000 7f3202782640 1 --2- 172.21.15.97:0/1581108958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31fc09ba40 0x7f31fc15cfd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f31fc072a80 tx=0x7f31f402bed0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.127+0000 7f3202782640 1 -- 172.21.15.97:0/1581108958 shutdown_connections 2024-03-29T04:08:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.127+0000 7f3202782640 1 --2- 172.21.15.97:0/1581108958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31fc09ba40 0x7f31fc15cfd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.127+0000 7f3202782640 1 --2- 172.21.15.97:0/1581108958 >> v2:172.21.15.154:3300/0 conn(0x7f31fc0d0a90 0x7f31fc0d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.127+0000 7f3202782640 1 -- 172.21.15.97:0/1581108958 >> 172.21.15.97:0/1581108958 conn(0x7f31fc0b3fa0 msgr2=0x7f31fc0b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.127+0000 7f3202782640 1 -- 172.21.15.97:0/1581108958 shutdown_connections 2024-03-29T04:08:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.127+0000 7f3202782640 1 -- 172.21.15.97:0/1581108958 wait complete. 2024-03-29T04:08:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.128+0000 7f3202782640 1 Processor -- start 2024-03-29T04:08:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.128+0000 7f3202782640 1 -- start start 2024-03-29T04:08:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f3202782640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31fc09ba40 0x7f31fc150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f3202782640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31fc0d0080 0x7f31fc0b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f3202782640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31fc0d0a90 0x7f31fc0b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f3202782640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc0ccfc0 con 0x7f31fc09ba40 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f3202782640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc0cdf90 con 0x7f31fc0d0080 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f3202782640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f31fc0cd140 con 0x7f31fc0d0a90 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f31fbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31fc09ba40 0x7f31fc150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f31fbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31fc09ba40 0x7f31fc150ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39396/0 (socket says 172.21.15.97:39396) 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f31fbfff640 1 -- 172.21.15.97:0/742612751 learned_addr learned my addr 172.21.15.97:0/742612751 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f3200d1f640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31fc0d0a90 0x7f31fc0b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f31fb7fe640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31fc0d0080 0x7f31fc0b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f31fbfff640 1 -- 172.21.15.97:0/742612751 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31fc0d0a90 msgr2=0x7f31fc0b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f31fbfff640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31fc0d0a90 0x7f31fc0b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f31fbfff640 1 -- 172.21.15.97:0/742612751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31fc0d0080 msgr2=0x7f31fc0b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f31fbfff640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31fc0d0080 0x7f31fc0b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f31fbfff640 1 -- 172.21.15.97:0/742612751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc0c80d0 con 0x7f31fc09ba40 2024-03-29T04:08:03.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f31fbfff640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31fc09ba40 0x7f31fc150ac0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00e8a0 tx=0x7f31ec00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f31f8ff9640 1 -- 172.21.15.97:0/742612751 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec017070 con 0x7f31fc09ba40 2024-03-29T04:08:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f31f8ff9640 1 -- 172.21.15.97:0/742612751 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ec00c940 con 0x7f31fc09ba40 2024-03-29T04:08:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f3202782640 1 -- 172.21.15.97:0/742612751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31fc0c83b0 con 0x7f31fc09ba40 2024-03-29T04:08:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f31f8ff9640 1 -- 172.21.15.97:0/742612751 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00cd30 con 0x7f31fc09ba40 2024-03-29T04:08:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.129+0000 7f3202782640 1 -- 172.21.15.97:0/742612751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31fc0c8970 con 0x7f31fc09ba40 2024-03-29T04:08:03.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.130+0000 7f3202782640 1 -- 172.21.15.97:0/742612751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31fc15c030 con 0x7f31fc09ba40 2024-03-29T04:08:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.131+0000 7f31f8ff9640 1 -- 172.21.15.97:0/742612751 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f31ec01a070 con 0x7f31fc09ba40 2024-03-29T04:08:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.131+0000 7f31f8ff9640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f31e806c530 0x7f31e806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.131+0000 7f31f8ff9640 1 -- 172.21.15.97:0/742612751 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f31ec08b7d0 con 0x7f31fc09ba40 2024-03-29T04:08:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.131+0000 7f31fb7fe640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f31e806c530 0x7f31e806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.131+0000 7f31fb7fe640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f31e806c530 0x7f31e806e9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0b5540 tx=0x7f31f0007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:03.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.135+0000 7f31f8ff9640 1 -- 172.21.15.97:0/742612751 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f31fc15c030 con 0x7f31fc09ba40 2024-03-29T04:08:03.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.309+0000 7f3202782640 1 -- 172.21.15.97:0/742612751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f31fc0a05c0 con 0x7f31fc09ba40 2024-03-29T04:08:03.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.310+0000 7f31f8ff9640 1 -- 172.21.15.97:0/742612751 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f31ec059830 con 0x7f31fc09ba40 2024-03-29T04:08:03.290 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:08:03.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.312+0000 7f3202782640 1 -- 172.21.15.97:0/742612751 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f31e806c530 msgr2=0x7f31e806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.312+0000 7f3202782640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f31e806c530 0x7f31e806e9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0b5540 tx=0x7f31f0007d50 comp rx=0 tx=0).stop 2024-03-29T04:08:03.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.312+0000 7f3202782640 1 -- 172.21.15.97:0/742612751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31fc09ba40 msgr2=0x7f31fc150ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.312+0000 7f3202782640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31fc09ba40 0x7f31fc150ac0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00e8a0 tx=0x7f31ec00ed60 comp rx=0 tx=0).stop 2024-03-29T04:08:03.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.312+0000 7f3202782640 1 -- 172.21.15.97:0/742612751 shutdown_connections 2024-03-29T04:08:03.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.312+0000 7f3202782640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f31e806c530 0x7f31e806e9f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.312+0000 7f3202782640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31fc0d0a90 0x7f31fc0b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.312+0000 7f3202782640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31fc0d0080 0x7f31fc0b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.312+0000 7f3202782640 1 --2- 172.21.15.97:0/742612751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31fc09ba40 0x7f31fc150ac0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.312+0000 7f3202782640 1 -- 172.21.15.97:0/742612751 >> 172.21.15.97:0/742612751 conn(0x7f31fc0b3fa0 msgr2=0x7f31fc15fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:03.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.312+0000 7f3202782640 1 -- 172.21.15.97:0/742612751 shutdown_connections 2024-03-29T04:08:03.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.312+0000 7f3202782640 1 -- 172.21.15.97:0/742612751 wait complete. 2024-03-29T04:08:03.302 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:03.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.422+0000 7f38cbb32640 1 Processor -- start 2024-03-29T04:08:03.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.423+0000 7f38cbb32640 1 -- start start 2024-03-29T04:08:03.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.423+0000 7f38cbb32640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f38c4144790 0x7f38c40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.423+0000 7f38cbb32640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38c406e1c0 0x7f38c406e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.423+0000 7f38cbb32640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38c40b3c10 con 0x7f38c406f1e0 2024-03-29T04:08:03.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.423+0000 7f38cbb32640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f38c40b39d0 con 0x7f38c406e1c0 2024-03-29T04:08:03.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.423+0000 7f38cbb32640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f38c40b3ef0 con 0x7f38c4144790 2024-03-29T04:08:03.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.424+0000 7f38ca0cf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38c406e1c0 0x7f38c406e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.424+0000 7f38c90cd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f38c406f1e0 0x7f38c40b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33456/0 (socket says 172.21.15.97:33456) 2024-03-29T04:08:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.424+0000 7f38c90cd640 1 -- 172.21.15.97:0/1867446157 learned_addr learned my addr 172.21.15.97:0/1867446157 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.424+0000 7f38c98ce640 1 --2- 172.21.15.97:0/1867446157 >> v2:172.21.15.154:3300/0 conn(0x7f38c4144790 0x7f38c40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.424+0000 7f38ca0cf640 1 -- 172.21.15.97:0/1867446157 >> v2:172.21.15.154:3300/0 conn(0x7f38c4144790 msgr2=0x7f38c40b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.424+0000 7f38ca0cf640 1 --2- 172.21.15.97:0/1867446157 >> v2:172.21.15.154:3300/0 conn(0x7f38c4144790 0x7f38c40b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.424+0000 7f38ca0cf640 1 -- 172.21.15.97:0/1867446157 >> v1:172.21.15.97:6789/0 conn(0x7f38c406f1e0 legacy=0x7f38c40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.424+0000 7f38c88cc640 1 -- 172.21.15.97:0/1867446157 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2318143433 0 0) 0x7f38c40b3c10 con 0x7f38c406f1e0 2024-03-29T04:08:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.424+0000 7f38ca0cf640 1 -- 172.21.15.97:0/1867446157 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c40b41d0 con 0x7f38c406e1c0 2024-03-29T04:08:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.424+0000 7f38c98ce640 1 --2- 172.21.15.97:0/1867446157 >> v2:172.21.15.154:3300/0 conn(0x7f38c4144790 0x7f38c40b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.425+0000 7f38ca0cf640 1 --2- 172.21.15.97:0/1867446157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c406e1c0 0x7f38c406e650 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f38c4070e80 tx=0x7f38bc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920fabddbd7ca98a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:03.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.425+0000 7f38c88cc640 1 -- 172.21.15.97:0/1867446157 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc039070 con 0x7f38c406e1c0 2024-03-29T04:08:03.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.425+0000 7f38c88cc640 1 -- 172.21.15.97:0/1867446157 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38bc02cc80 con 0x7f38c406e1c0 2024-03-29T04:08:03.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.425+0000 7f38c88cc640 1 -- 172.21.15.97:0/1867446157 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc034660 con 0x7f38c406e1c0 2024-03-29T04:08:03.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.425+0000 7f38cbb32640 1 -- 172.21.15.97:0/1867446157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c406e1c0 msgr2=0x7f38c406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.425+0000 7f38cbb32640 1 --2- 172.21.15.97:0/1867446157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c406e1c0 0x7f38c406e650 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f38c4070e80 tx=0x7f38bc02bed0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.425+0000 7f38cbb32640 1 -- 172.21.15.97:0/1867446157 shutdown_connections 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.425+0000 7f38cbb32640 1 --2- 172.21.15.97:0/1867446157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c406e1c0 0x7f38c406e650 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.425+0000 7f38cbb32640 1 --2- 172.21.15.97:0/1867446157 >> v2:172.21.15.154:3300/0 conn(0x7f38c4144790 0x7f38c40b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.425+0000 7f38cbb32640 1 -- 172.21.15.97:0/1867446157 >> 172.21.15.97:0/1867446157 conn(0x7f38c406b510 msgr2=0x7f38c406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.425+0000 7f38cbb32640 1 -- 172.21.15.97:0/1867446157 shutdown_connections 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.425+0000 7f38cbb32640 1 -- 172.21.15.97:0/1867446157 wait complete. 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.427+0000 7f38cbb32640 1 Processor -- start 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.427+0000 7f38cbb32640 1 -- start start 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.427+0000 7f38cbb32640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38c406e1c0 0x7f38c41558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.427+0000 7f38cbb32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c406f1e0 0x7f38c4159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.427+0000 7f38cbb32640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38c4144790 0x7f38c4166970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.427+0000 7f38cbb32640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f38c40b51f0 con 0x7f38c406f1e0 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.427+0000 7f38cbb32640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f38c40b4790 con 0x7f38c4144790 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.427+0000 7f38cbb32640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f38c415aa00 con 0x7f38c406e1c0 2024-03-29T04:08:03.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.427+0000 7f38c90cd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c406f1e0 0x7f38c4159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.427+0000 7f38c90cd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c406f1e0 0x7f38c4159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39412/0 (socket says 172.21.15.97:39412) 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.427+0000 7f38c90cd640 1 -- 172.21.15.97:0/3930331155 learned_addr learned my addr 172.21.15.97:0/3930331155 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.427+0000 7f38c98ce640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38c406e1c0 0x7f38c41558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.427+0000 7f38ca0cf640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38c4144790 0x7f38c4166970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.428+0000 7f38c90cd640 1 -- 172.21.15.97:0/3930331155 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38c406e1c0 msgr2=0x7f38c41558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.428+0000 7f38c90cd640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38c406e1c0 0x7f38c41558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.428+0000 7f38c90cd640 1 -- 172.21.15.97:0/3930331155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38c4144790 msgr2=0x7f38c4166970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.428+0000 7f38c90cd640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38c4144790 0x7f38c4166970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.428+0000 7f38c90cd640 1 -- 172.21.15.97:0/3930331155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c4155e30 con 0x7f38c406f1e0 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.428+0000 7f38c90cd640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c406f1e0 0x7f38c4159410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38c000c460 tx=0x7f38c000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.428+0000 7f38b67fc640 1 -- 172.21.15.97:0/3930331155 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0018070 con 0x7f38c406f1e0 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.428+0000 7f38b67fc640 1 -- 172.21.15.97:0/3930331155 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c0007520 con 0x7f38c406f1e0 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.428+0000 7f38b67fc640 1 -- 172.21.15.97:0/3930331155 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0004a20 con 0x7f38c406f1e0 2024-03-29T04:08:03.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.428+0000 7f38cbb32640 1 -- 172.21.15.97:0/3930331155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c4159df0 con 0x7f38c406f1e0 2024-03-29T04:08:03.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.428+0000 7f38cbb32640 1 -- 172.21.15.97:0/3930331155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c415a3b0 con 0x7f38c406f1e0 2024-03-29T04:08:03.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.429+0000 7f38cbb32640 1 -- 172.21.15.97:0/3930331155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c4068640 con 0x7f38c406f1e0 2024-03-29T04:08:03.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.429+0000 7f38b67fc640 1 -- 172.21.15.97:0/3930331155 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f38c001b070 con 0x7f38c406f1e0 2024-03-29T04:08:03.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.430+0000 7f38b67fc640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f38a806c530 0x7f38a806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.430+0000 7f38b67fc640 1 -- 172.21.15.97:0/3930331155 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f38c008a9d0 con 0x7f38c406f1e0 2024-03-29T04:08:03.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.430+0000 7f38c98ce640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f38a806c530 0x7f38a806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.430+0000 7f38c98ce640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f38a806c530 0x7f38a806e9f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009560 tx=0x7f38b8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:03.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.434+0000 7f38b67fc640 1 -- 172.21.15.97:0/3930331155 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f38c00589a0 con 0x7f38c406f1e0 2024-03-29T04:08:03.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.610+0000 7f38cbb32640 1 -- 172.21.15.97:0/3930331155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38c415a750 con 0x7f38c406f1e0 2024-03-29T04:08:03.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.611+0000 7f38b67fc640 1 -- 172.21.15.97:0/3930331155 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f38c005cca0 con 0x7f38c406f1e0 2024-03-29T04:08:03.591 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:03.591 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.613+0000 7f38cbb32640 1 -- 172.21.15.97:0/3930331155 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f38a806c530 msgr2=0x7f38a806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.613+0000 7f38cbb32640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f38a806c530 0x7f38a806e9f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009560 tx=0x7f38b8009190 comp rx=0 tx=0).stop 2024-03-29T04:08:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.613+0000 7f38cbb32640 1 -- 172.21.15.97:0/3930331155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c406f1e0 msgr2=0x7f38c4159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.613+0000 7f38cbb32640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c406f1e0 0x7f38c4159410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38c000c460 tx=0x7f38c000c920 comp rx=0 tx=0).stop 2024-03-29T04:08:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.613+0000 7f38cbb32640 1 -- 172.21.15.97:0/3930331155 shutdown_connections 2024-03-29T04:08:03.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.613+0000 7f38cbb32640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f38a806c530 0x7f38a806e9f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.613+0000 7f38cbb32640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38c4144790 0x7f38c4166970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.613+0000 7f38cbb32640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c406f1e0 0x7f38c4159410 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.613+0000 7f38cbb32640 1 --2- 172.21.15.97:0/3930331155 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f38c406e1c0 0x7f38c41558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.613+0000 7f38cbb32640 1 -- 172.21.15.97:0/3930331155 >> 172.21.15.97:0/3930331155 conn(0x7f38c406b510 msgr2=0x7f38c40b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.613+0000 7f38cbb32640 1 -- 172.21.15.97:0/3930331155 shutdown_connections 2024-03-29T04:08:03.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.613+0000 7f38cbb32640 1 -- 172.21.15.97:0/3930331155 wait complete. 2024-03-29T04:08:03.604 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:07:38.618704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:07:39.724339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:08:03.604 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:08:03.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.727+0000 7f7ae9324640 1 Processor -- start 2024-03-29T04:08:03.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.727+0000 7f7ae9324640 1 -- start start 2024-03-29T04:08:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.728+0000 7f7ae9324640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae41624b0 0x7f7ae40b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.728+0000 7f7ae9324640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ae40ccea0 0x7f7ae4162890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.728+0000 7f7ae9324640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ae406d1b0 con 0x7f7ae40cca60 2024-03-29T04:08:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.728+0000 7f7ae9324640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae40d0a10 con 0x7f7ae40ccea0 2024-03-29T04:08:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.728+0000 7f7ae9324640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae406d470 con 0x7f7ae41624b0 2024-03-29T04:08:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.728+0000 7f7ae27fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7ae40cca60 0x7f7ae40d0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33464/0 (socket says 172.21.15.97:33464) 2024-03-29T04:08:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.728+0000 7f7ae27fc640 1 -- 172.21.15.97:0/1222716615 learned_addr learned my addr 172.21.15.97:0/1222716615 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.728+0000 7f7ae2ffd640 1 --2- 172.21.15.97:0/1222716615 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae41624b0 0x7f7ae40b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.728+0000 7f7ae37fe640 1 --2- 172.21.15.97:0/1222716615 >> v2:172.21.15.154:3300/0 conn(0x7f7ae40ccea0 0x7f7ae4162890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.728+0000 7f7ae1ffb640 1 -- 172.21.15.97:0/1222716615 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1196829694 0 0) 0x7f7ae406d1b0 con 0x7f7ae40cca60 2024-03-29T04:08:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.728+0000 7f7ae1ffb640 1 -- 172.21.15.97:0/1222716615 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7acc003520 con 0x7f7ae40cca60 2024-03-29T04:08:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.729+0000 7f7ae2ffd640 1 -- 172.21.15.97:0/1222716615 >> v2:172.21.15.154:3300/0 conn(0x7f7ae40ccea0 msgr2=0x7f7ae4162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:03.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.729+0000 7f7ae2ffd640 1 --2- 172.21.15.97:0/1222716615 >> v2:172.21.15.154:3300/0 conn(0x7f7ae40ccea0 0x7f7ae4162890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.729+0000 7f7ae2ffd640 1 -- 172.21.15.97:0/1222716615 >> v1:172.21.15.97:6789/0 conn(0x7f7ae40cca60 legacy=0x7f7ae40d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.729+0000 7f7ae2ffd640 1 -- 172.21.15.97:0/1222716615 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae406d750 con 0x7f7ae41624b0 2024-03-29T04:08:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.729+0000 7f7ae37fe640 1 --2- 172.21.15.97:0/1222716615 >> v2:172.21.15.154:3300/0 conn(0x7f7ae40ccea0 0x7f7ae4162890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.729+0000 7f7ae2ffd640 1 --2- 172.21.15.97:0/1222716615 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae41624b0 0x7f7ae40b33b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009d90 tx=0x7f7ad402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc06fafe2c783b40 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:03.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.729+0000 7f7ae1ffb640 1 -- 172.21.15.97:0/1222716615 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4039070 con 0x7f7ae41624b0 2024-03-29T04:08:03.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.730+0000 7f7ae1ffb640 1 -- 172.21.15.97:0/1222716615 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad402cc50 con 0x7f7ae41624b0 2024-03-29T04:08:03.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.730+0000 7f7ae1ffb640 1 -- 172.21.15.97:0/1222716615 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4034440 con 0x7f7ae41624b0 2024-03-29T04:08:03.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.730+0000 7f7ae9324640 1 -- 172.21.15.97:0/1222716615 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae41624b0 msgr2=0x7f7ae40b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.730+0000 7f7ae9324640 1 --2- 172.21.15.97:0/1222716615 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae41624b0 0x7f7ae40b33b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009d90 tx=0x7f7ad402bea0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.730+0000 7f7ae9324640 1 -- 172.21.15.97:0/1222716615 shutdown_connections 2024-03-29T04:08:03.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.730+0000 7f7ae9324640 1 --2- 172.21.15.97:0/1222716615 >> v2:172.21.15.154:3300/0 conn(0x7f7ae40ccea0 0x7f7ae4162890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.730+0000 7f7ae9324640 1 --2- 172.21.15.97:0/1222716615 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae41624b0 0x7f7ae40b33b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.730+0000 7f7ae9324640 1 -- 172.21.15.97:0/1222716615 >> 172.21.15.97:0/1222716615 conn(0x7f7ae40b3f00 msgr2=0x7f7ae40b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.730+0000 7f7ae9324640 1 -- 172.21.15.97:0/1222716615 shutdown_connections 2024-03-29T04:08:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.730+0000 7f7ae9324640 1 -- 172.21.15.97:0/1222716615 wait complete. 2024-03-29T04:08:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.731+0000 7f7ae9324640 1 Processor -- start 2024-03-29T04:08:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.731+0000 7f7ae9324640 1 -- start start 2024-03-29T04:08:03.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae9324640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae40cca60 0x7f7ae4151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae9324640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae40ccea0 0x7f7ae41f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae9324640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ae41624b0 0x7f7ae40c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae9324640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae415feb0 con 0x7f7ae41624b0 2024-03-29T04:08:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae9324640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae415fd30 con 0x7f7ae40cca60 2024-03-29T04:08:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae9324640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae40d15b0 con 0x7f7ae40ccea0 2024-03-29T04:08:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ae41624b0 0x7f7ae40c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ae41624b0 0x7f7ae40c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39426/0 (socket says 172.21.15.97:39426) 2024-03-29T04:08:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae37fe640 1 -- 172.21.15.97:0/2784563588 learned_addr learned my addr 172.21.15.97:0/2784563588 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae2ffd640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae40cca60 0x7f7ae4151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae27fc640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae40ccea0 0x7f7ae41f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae37fe640 1 -- 172.21.15.97:0/2784563588 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae40ccea0 msgr2=0x7f7ae41f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae37fe640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae40ccea0 0x7f7ae41f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae37fe640 1 -- 172.21.15.97:0/2784563588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae40cca60 msgr2=0x7f7ae4151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae37fe640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae40cca60 0x7f7ae4151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.732+0000 7f7ae37fe640 1 -- 172.21.15.97:0/2784563588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae40c6190 con 0x7f7ae41624b0 2024-03-29T04:08:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.733+0000 7f7ae37fe640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ae41624b0 0x7f7ae40c82c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00e900 tx=0x7f7adc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.733+0000 7f7acb7fe640 1 -- 172.21.15.97:0/2784563588 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc017070 con 0x7f7ae41624b0 2024-03-29T04:08:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.733+0000 7f7acb7fe640 1 -- 172.21.15.97:0/2784563588 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7adc00ca00 con 0x7f7ae41624b0 2024-03-29T04:08:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.733+0000 7f7ae9324640 1 -- 172.21.15.97:0/2784563588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae40c6470 con 0x7f7ae41624b0 2024-03-29T04:08:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.733+0000 7f7acb7fe640 1 -- 172.21.15.97:0/2784563588 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00cdb0 con 0x7f7ae41624b0 2024-03-29T04:08:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.733+0000 7f7ae9324640 1 -- 172.21.15.97:0/2784563588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae40c6a30 con 0x7f7ae41624b0 2024-03-29T04:08:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.734+0000 7f7ae9324640 1 -- 172.21.15.97:0/2784563588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae40c6c30 con 0x7f7ae41624b0 2024-03-29T04:08:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.734+0000 7f7acb7fe640 1 -- 172.21.15.97:0/2784563588 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7adc01a070 con 0x7f7ae41624b0 2024-03-29T04:08:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.735+0000 7f7acb7fe640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ad006c530 0x7f7ad006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.735+0000 7f7ae2ffd640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ad006c530 0x7f7ad006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:03.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.735+0000 7f7acb7fe640 1 -- 172.21.15.97:0/2784563588 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7adc08b8b0 con 0x7f7ae41624b0 2024-03-29T04:08:03.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.735+0000 7f7ae2ffd640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ad006c530 0x7f7ad006e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4036000 tx=0x7f7ad4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:03.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.739+0000 7f7acb7fe640 1 -- 172.21.15.97:0/2784563588 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7ae40c6c30 con 0x7f7ae41624b0 2024-03-29T04:08:03.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.913+0000 7f7ae9324640 1 -- 172.21.15.97:0/2784563588 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7ae40a04c0 con 0x7f7ae41624b0 2024-03-29T04:08:03.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.913+0000 7f7acb7fe640 1 -- 172.21.15.97:0/2784563588 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f7adc059880 con 0x7f7ae41624b0 2024-03-29T04:08:03.893 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:08:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.915+0000 7f7ae9324640 1 -- 172.21.15.97:0/2784563588 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ad006c530 msgr2=0x7f7ad006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.915+0000 7f7ae9324640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ad006c530 0x7f7ad006e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4036000 tx=0x7f7ad4030000 comp rx=0 tx=0).stop 2024-03-29T04:08:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.915+0000 7f7ae9324640 1 -- 172.21.15.97:0/2784563588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ae41624b0 msgr2=0x7f7ae40c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.915+0000 7f7ae9324640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ae41624b0 0x7f7ae40c82c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00e900 tx=0x7f7adc00edc0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.915+0000 7f7ae9324640 1 -- 172.21.15.97:0/2784563588 shutdown_connections 2024-03-29T04:08:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.915+0000 7f7ae9324640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ad006c530 0x7f7ad006e9f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.915+0000 7f7ae9324640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ae41624b0 0x7f7ae40c82c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.915+0000 7f7ae9324640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae40ccea0 0x7f7ae41f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.915+0000 7f7ae9324640 1 --2- 172.21.15.97:0/2784563588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae40cca60 0x7f7ae4151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.915+0000 7f7ae9324640 1 -- 172.21.15.97:0/2784563588 >> 172.21.15.97:0/2784563588 conn(0x7f7ae40b3f00 msgr2=0x7f7ae40d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.915+0000 7f7ae9324640 1 -- 172.21.15.97:0/2784563588 shutdown_connections 2024-03-29T04:08:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:03.915+0000 7f7ae9324640 1 -- 172.21.15.97:0/2784563588 wait complete. 2024-03-29T04:08:03.906 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:08:04.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.029+0000 7f319c182640 1 Processor -- start 2024-03-29T04:08:04.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.029+0000 7f319c182640 1 -- start start 2024-03-29T04:08:04.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.030+0000 7f319c182640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f319409aa30 0x7f319409aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.030+0000 7f319c182640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31940d06f0 con 0x7f31940d0bd0 2024-03-29T04:08:04.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.030+0000 7f319c182640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3194070250 con 0x7f3194161f50 2024-03-29T04:08:04.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.030+0000 7f319c182640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f319415f300 con 0x7f319409aa30 2024-03-29T04:08:04.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.030+0000 7f319a71f640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f31940d0bd0 0x7f31940d0fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33470/0 (socket says 172.21.15.97:33470) 2024-03-29T04:08:04.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.030+0000 7f319a71f640 1 -- 172.21.15.97:0/2250481059 learned_addr learned my addr 172.21.15.97:0/2250481059 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:04.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.030+0000 7f319971d640 1 --2- 172.21.15.97:0/2250481059 >> v2:172.21.15.154:3300/0 conn(0x7f319409aa30 0x7f319409aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.030+0000 7f3198f1c640 1 -- 172.21.15.97:0/2250481059 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2773746897 0 0) 0x7f31940d06f0 con 0x7f31940d0bd0 2024-03-29T04:08:04.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.031+0000 7f3198f1c640 1 -- 172.21.15.97:0/2250481059 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3180003520 con 0x7f31940d0bd0 2024-03-29T04:08:04.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.031+0000 7f3198f1c640 1 -- 172.21.15.97:0/2250481059 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3956774708 0 0) 0x7f3194070250 con 0x7f3194161f50 2024-03-29T04:08:04.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.031+0000 7f319971d640 1 -- 172.21.15.97:0/2250481059 >> v1:172.21.15.154:6789/0 conn(0x7f3194161f50 legacy=0x7f319406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.031+0000 7f319971d640 1 -- 172.21.15.97:0/2250481059 >> v1:172.21.15.97:6789/0 conn(0x7f31940d0bd0 legacy=0x7f31940d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.031+0000 7f319971d640 1 -- 172.21.15.97:0/2250481059 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319415f5e0 con 0x7f319409aa30 2024-03-29T04:08:04.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.031+0000 7f319971d640 1 --2- 172.21.15.97:0/2250481059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f319409aa30 0x7f319409aec0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f31840097a0 tx=0x7f318402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391bd85741c20dac server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:04.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.031+0000 7f3198f1c640 1 -- 172.21.15.97:0/2250481059 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184035020 con 0x7f319409aa30 2024-03-29T04:08:04.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.032+0000 7f3198f1c640 1 -- 172.21.15.97:0/2250481059 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3184031830 con 0x7f319409aa30 2024-03-29T04:08:04.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.032+0000 7f3198f1c640 1 -- 172.21.15.97:0/2250481059 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184031c00 con 0x7f319409aa30 2024-03-29T04:08:04.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.032+0000 7f319c182640 1 -- 172.21.15.97:0/2250481059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f319409aa30 msgr2=0x7f319409aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.032+0000 7f319c182640 1 --2- 172.21.15.97:0/2250481059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f319409aa30 0x7f319409aec0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f31840097a0 tx=0x7f318402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.032+0000 7f319c182640 1 -- 172.21.15.97:0/2250481059 shutdown_connections 2024-03-29T04:08:04.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.032+0000 7f319c182640 1 --2- 172.21.15.97:0/2250481059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f319409aa30 0x7f319409aec0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.032+0000 7f319c182640 1 -- 172.21.15.97:0/2250481059 >> 172.21.15.97:0/2250481059 conn(0x7f31940b3fa0 msgr2=0x7f31940b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:04.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.032+0000 7f319c182640 1 -- 172.21.15.97:0/2250481059 shutdown_connections 2024-03-29T04:08:04.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.032+0000 7f319c182640 1 -- 172.21.15.97:0/2250481059 wait complete. 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.033+0000 7f319c182640 1 Processor -- start 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.033+0000 7f319c182640 1 -- start start 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f319c182640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319409aa30 0x7f3194159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f319c182640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31940d0bd0 0x7f319415b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f319c182640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3194161f50 0x7f319415cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f319c182640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3194160220 con 0x7f319409aa30 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f319c182640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f319415fba0 con 0x7f31940d0bd0 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f319c182640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f31941603a0 con 0x7f3194161f50 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f3199f1e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319409aa30 0x7f3194159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f3199f1e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319409aa30 0x7f3194159670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39432/0 (socket says 172.21.15.97:39432) 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f3199f1e640 1 -- 172.21.15.97:0/2364465549 learned_addr learned my addr 172.21.15.97:0/2364465549 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f319971d640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31940d0bd0 0x7f319415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f319a71f640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3194161f50 0x7f319415cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f3199f1e640 1 -- 172.21.15.97:0/2364465549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3194161f50 msgr2=0x7f319415cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f3199f1e640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3194161f50 0x7f319415cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f3199f1e640 1 -- 172.21.15.97:0/2364465549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31940d0bd0 msgr2=0x7f319415b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f3199f1e640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31940d0bd0 0x7f319415b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.034+0000 7f3199f1e640 1 -- 172.21.15.97:0/2364465549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319415a5d0 con 0x7f319409aa30 2024-03-29T04:08:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.035+0000 7f3199f1e640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319409aa30 0x7f3194159670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f319000c4b0 tx=0x7f319000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.035+0000 7f318a7fc640 1 -- 172.21.15.97:0/2364465549 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190017070 con 0x7f319409aa30 2024-03-29T04:08:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.035+0000 7f318a7fc640 1 -- 172.21.15.97:0/2364465549 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3190005480 con 0x7f319409aa30 2024-03-29T04:08:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.035+0000 7f319c182640 1 -- 172.21.15.97:0/2364465549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319415a8b0 con 0x7f319409aa30 2024-03-29T04:08:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.035+0000 7f318a7fc640 1 -- 172.21.15.97:0/2364465549 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190003830 con 0x7f319409aa30 2024-03-29T04:08:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.035+0000 7f319c182640 1 -- 172.21.15.97:0/2364465549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319415d700 con 0x7f319409aa30 2024-03-29T04:08:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.036+0000 7f319c182640 1 -- 172.21.15.97:0/2364465549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31940a05c0 con 0x7f319409aa30 2024-03-29T04:08:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.036+0000 7f318a7fc640 1 -- 172.21.15.97:0/2364465549 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f319001a070 con 0x7f319409aa30 2024-03-29T04:08:04.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.036+0000 7f318a7fc640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f317806c560 0x7f317806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.037+0000 7f319971d640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f317806c560 0x7f317806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.037+0000 7f318a7fc640 1 -- 172.21.15.97:0/2364465549 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f319008a970 con 0x7f319409aa30 2024-03-29T04:08:04.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.037+0000 7f319971d640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f317806c560 0x7f317806ea20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3184036000 tx=0x7f3184008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:04.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.041+0000 7f318a7fc640 1 -- 172.21.15.97:0/2364465549 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f31900589d0 con 0x7f319409aa30 2024-03-29T04:08:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.213+0000 7f319c182640 1 -- 172.21.15.97:0/2364465549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f319404f9d0 con 0x7f319409aa30 2024-03-29T04:08:04.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.214+0000 7f318a7fc640 1 -- 172.21.15.97:0/2364465549 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f319005ccd0 con 0x7f319409aa30 2024-03-29T04:08:04.193 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:08:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.216+0000 7f319c182640 1 -- 172.21.15.97:0/2364465549 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f317806c560 msgr2=0x7f317806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.216+0000 7f319c182640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f317806c560 0x7f317806ea20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3184036000 tx=0x7f3184008030 comp rx=0 tx=0).stop 2024-03-29T04:08:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.216+0000 7f319c182640 1 -- 172.21.15.97:0/2364465549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319409aa30 msgr2=0x7f3194159670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.216+0000 7f319c182640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319409aa30 0x7f3194159670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f319000c4b0 tx=0x7f319000c970 comp rx=0 tx=0).stop 2024-03-29T04:08:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.216+0000 7f319c182640 1 -- 172.21.15.97:0/2364465549 shutdown_connections 2024-03-29T04:08:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.216+0000 7f319c182640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f317806c560 0x7f317806ea20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.216+0000 7f319c182640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3194161f50 0x7f319415cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.216+0000 7f319c182640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31940d0bd0 0x7f319415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.216+0000 7f319c182640 1 --2- 172.21.15.97:0/2364465549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f319409aa30 0x7f3194159670 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.216+0000 7f319c182640 1 -- 172.21.15.97:0/2364465549 >> 172.21.15.97:0/2364465549 conn(0x7f31940b3fa0 msgr2=0x7f31940b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:04.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.216+0000 7f319c182640 1 -- 172.21.15.97:0/2364465549 shutdown_connections 2024-03-29T04:08:04.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.216+0000 7f319c182640 1 -- 172.21.15.97:0/2364465549 wait complete. 2024-03-29T04:08:04.206 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:08:04.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.330+0000 7f9971d24640 1 Processor -- start 2024-03-29T04:08:04.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.330+0000 7f9971d24640 1 -- start start 2024-03-29T04:08:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f9971d24640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f996c0722e0 0x7f996c06ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f9971d24640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f996c147c90 0x7f996c06f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f9971d24640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f996c06df60 con 0x7f996c1477a0 2024-03-29T04:08:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f9971d24640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f996c06e830 con 0x7f996c0722e0 2024-03-29T04:08:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f9971d24640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f996c162e60 con 0x7f996c147c90 2024-03-29T04:08:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f996affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f996c0722e0 0x7f996c06ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f996b7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f996c1477a0 0x7f996c071bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33478/0 (socket says 172.21.15.97:33478) 2024-03-29T04:08:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f996b7fe640 1 -- 172.21.15.97:0/3791792725 learned_addr learned my addr 172.21.15.97:0/3791792725 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f996bfff640 1 --2- 172.21.15.97:0/3791792725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f996c147c90 0x7f996c06f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f996affd640 1 -- 172.21.15.97:0/3791792725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f996c147c90 msgr2=0x7f996c06f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:04.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f996affd640 1 --2- 172.21.15.97:0/3791792725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f996c147c90 0x7f996c06f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f996affd640 1 -- 172.21.15.97:0/3791792725 >> v1:172.21.15.97:6789/0 conn(0x7f996c1477a0 legacy=0x7f996c071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f996affd640 1 -- 172.21.15.97:0/3791792725 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c163140 con 0x7f996c0722e0 2024-03-29T04:08:04.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.331+0000 7f996a7fc640 1 -- 172.21.15.97:0/3791792725 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1433608982 0 0) 0x7f995c001110 con 0x7f996c1477a0 2024-03-29T04:08:04.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.332+0000 7f996affd640 1 --2- 172.21.15.97:0/3791792725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996c0722e0 0x7f996c06ecc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f99600097a0 tx=0x7f996002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e889a21525bd59bb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:04.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.332+0000 7f996a7fc640 1 -- 172.21.15.97:0/3791792725 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960035020 con 0x7f996c0722e0 2024-03-29T04:08:04.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.332+0000 7f996a7fc640 1 -- 172.21.15.97:0/3791792725 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9960031830 con 0x7f996c0722e0 2024-03-29T04:08:04.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.332+0000 7f996a7fc640 1 -- 172.21.15.97:0/3791792725 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960031c00 con 0x7f996c0722e0 2024-03-29T04:08:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.333+0000 7f9971d24640 1 -- 172.21.15.97:0/3791792725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996c0722e0 msgr2=0x7f996c06ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.333+0000 7f9971d24640 1 --2- 172.21.15.97:0/3791792725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996c0722e0 0x7f996c06ecc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f99600097a0 tx=0x7f996002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.333+0000 7f9971d24640 1 -- 172.21.15.97:0/3791792725 shutdown_connections 2024-03-29T04:08:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.333+0000 7f9971d24640 1 --2- 172.21.15.97:0/3791792725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f996c147c90 0x7f996c06f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.333+0000 7f9971d24640 1 --2- 172.21.15.97:0/3791792725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996c0722e0 0x7f996c06ecc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.333+0000 7f9971d24640 1 -- 172.21.15.97:0/3791792725 >> 172.21.15.97:0/3791792725 conn(0x7f996c0b2340 msgr2=0x7f996c0b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.333+0000 7f9971d24640 1 -- 172.21.15.97:0/3791792725 shutdown_connections 2024-03-29T04:08:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.333+0000 7f9971d24640 1 -- 172.21.15.97:0/3791792725 wait complete. 2024-03-29T04:08:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.334+0000 7f9971d24640 1 Processor -- start 2024-03-29T04:08:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.334+0000 7f9971d24640 1 -- start start 2024-03-29T04:08:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f9971d24640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0722e0 0x7f996c1f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f9971d24640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996c1477a0 0x7f996c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f9971d24640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f996c147c90 0x7f996c13f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f9971d24640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1481e0 con 0x7f996c1477a0 2024-03-29T04:08:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f9971d24640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f996c06e4c0 con 0x7f996c0722e0 2024-03-29T04:08:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f9971d24640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f996c148360 con 0x7f996c147c90 2024-03-29T04:08:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f996affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996c1477a0 0x7f996c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f996affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996c1477a0 0x7f996c15e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39458/0 (socket says 172.21.15.97:39458) 2024-03-29T04:08:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f996affd640 1 -- 172.21.15.97:0/2917892295 learned_addr learned my addr 172.21.15.97:0/2917892295 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f996bfff640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f996c147c90 0x7f996c13f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f996b7fe640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0722e0 0x7f996c1f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f996affd640 1 -- 172.21.15.97:0/2917892295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f996c147c90 msgr2=0x7f996c13f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f996affd640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f996c147c90 0x7f996c13f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f996affd640 1 -- 172.21.15.97:0/2917892295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0722e0 msgr2=0x7f996c1f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f996affd640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0722e0 0x7f996c1f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f996affd640 1 -- 172.21.15.97:0/2917892295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c1f5c00 con 0x7f996c1477a0 2024-03-29T04:08:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f996affd640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996c1477a0 0x7f996c15e1a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f996002bee0 tx=0x7f99600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f9953fff640 1 -- 172.21.15.97:0/2917892295 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960035040 con 0x7f996c1477a0 2024-03-29T04:08:04.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f9953fff640 1 -- 172.21.15.97:0/2917892295 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996003b920 con 0x7f996c1477a0 2024-03-29T04:08:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f9971d24640 1 -- 172.21.15.97:0/2917892295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996c1451f0 con 0x7f996c1477a0 2024-03-29T04:08:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f9953fff640 1 -- 172.21.15.97:0/2917892295 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996003bd10 con 0x7f996c1477a0 2024-03-29T04:08:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.335+0000 7f9971d24640 1 -- 172.21.15.97:0/2917892295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996c1457b0 con 0x7f996c1477a0 2024-03-29T04:08:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.336+0000 7f9971d24640 1 -- 172.21.15.97:0/2917892295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f996c073580 con 0x7f996c1477a0 2024-03-29T04:08:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.337+0000 7f9953fff640 1 -- 172.21.15.97:0/2917892295 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9960038020 con 0x7f996c1477a0 2024-03-29T04:08:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.337+0000 7f9953fff640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f995806c530 0x7f995806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.337+0000 7f9953fff640 1 -- 172.21.15.97:0/2917892295 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f99600ad6c0 con 0x7f996c1477a0 2024-03-29T04:08:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.337+0000 7f996b7fe640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f995806c530 0x7f995806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.338+0000 7f996b7fe640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f995806c530 0x7f995806e9f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f995c000f60 tx=0x7f995c007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:04.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.342+0000 7f9953fff640 1 -- 172.21.15.97:0/2917892295 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9960031800 con 0x7f996c1477a0 2024-03-29T04:08:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.519+0000 7f9971d24640 1 -- 172.21.15.97:0/2917892295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f996c0727d0 con 0x7f996c1477a0 2024-03-29T04:08:04.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.519+0000 7f9953fff640 1 -- 172.21.15.97:0/2917892295 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f996007b6a0 con 0x7f996c1477a0 2024-03-29T04:08:04.499 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:08:04.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.521+0000 7f9971d24640 1 -- 172.21.15.97:0/2917892295 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f995806c530 msgr2=0x7f995806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.521+0000 7f9971d24640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f995806c530 0x7f995806e9f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f995c000f60 tx=0x7f995c007b50 comp rx=0 tx=0).stop 2024-03-29T04:08:04.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.521+0000 7f9971d24640 1 -- 172.21.15.97:0/2917892295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996c1477a0 msgr2=0x7f996c15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.521+0000 7f9971d24640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996c1477a0 0x7f996c15e1a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f996002bee0 tx=0x7f99600046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.521+0000 7f9971d24640 1 -- 172.21.15.97:0/2917892295 shutdown_connections 2024-03-29T04:08:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.521+0000 7f9971d24640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f995806c530 0x7f995806e9f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.521+0000 7f9971d24640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f996c147c90 0x7f996c13f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.521+0000 7f9971d24640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f996c1477a0 0x7f996c15e1a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.521+0000 7f9971d24640 1 --2- 172.21.15.97:0/2917892295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0722e0 0x7f996c1f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.521+0000 7f9971d24640 1 -- 172.21.15.97:0/2917892295 >> 172.21.15.97:0/2917892295 conn(0x7f996c0b2340 msgr2=0x7f996c0b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.522+0000 7f9971d24640 1 -- 172.21.15.97:0/2917892295 shutdown_connections 2024-03-29T04:08:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.522+0000 7f9971d24640 1 -- 172.21.15.97:0/2917892295 wait complete. 2024-03-29T04:08:04.512 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:04.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.633+0000 7ff330ac8640 1 Processor -- start 2024-03-29T04:08:04.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.633+0000 7ff330ac8640 1 -- start start 2024-03-29T04:08:04.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff330ac8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff32c15e880 0x7ff32c0cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff330ac8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff32c15ec60 0x7ff32c0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff330ac8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff32c0cf7f0 con 0x7ff32c0b4340 2024-03-29T04:08:04.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff330ac8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff32c0cfe00 con 0x7ff32c15ec60 2024-03-29T04:08:04.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff330ac8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ff32c0ce7c0 con 0x7ff32c15e880 2024-03-29T04:08:04.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff323fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff32c15ec60 0x7ff32c0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff32ad9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff32c0b4340 0x7ff32c06b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33488/0 (socket says 172.21.15.97:33488) 2024-03-29T04:08:04.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff32ad9d640 1 -- 172.21.15.97:0/3141218509 learned_addr learned my addr 172.21.15.97:0/3141218509 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:04.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff32a59c640 1 --2- 172.21.15.97:0/3141218509 >> v2:172.21.15.154:3300/0 conn(0x7ff32c15e880 0x7ff32c0cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff323fff640 1 -- 172.21.15.97:0/3141218509 >> v2:172.21.15.154:3300/0 conn(0x7ff32c15e880 msgr2=0x7ff32c0cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:04.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff323fff640 1 --2- 172.21.15.97:0/3141218509 >> v2:172.21.15.154:3300/0 conn(0x7ff32c15e880 0x7ff32c0cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff323fff640 1 -- 172.21.15.97:0/3141218509 >> v1:172.21.15.97:6789/0 conn(0x7ff32c0b4340 legacy=0x7ff32c06b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff329d9b640 1 -- 172.21.15.97:0/3141218509 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3087421962 0 0) 0x7ff32c0cf7f0 con 0x7ff32c0b4340 2024-03-29T04:08:04.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.634+0000 7ff323fff640 1 -- 172.21.15.97:0/3141218509 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32c0cea70 con 0x7ff32c15ec60 2024-03-29T04:08:04.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.635+0000 7ff323fff640 1 --2- 172.21.15.97:0/3141218509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff32c15ec60 0x7ff32c0ce280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff32c071390 tx=0x7ff31402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861074fbaca0d842 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:04.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.635+0000 7ff329d9b640 1 -- 172.21.15.97:0/3141218509 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3140315a0 con 0x7ff32c15ec60 2024-03-29T04:08:04.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.635+0000 7ff329d9b640 1 -- 172.21.15.97:0/3141218509 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff314037070 con 0x7ff32c15ec60 2024-03-29T04:08:04.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.635+0000 7ff329d9b640 1 -- 172.21.15.97:0/3141218509 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314031960 con 0x7ff32c15ec60 2024-03-29T04:08:04.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.635+0000 7ff330ac8640 1 -- 172.21.15.97:0/3141218509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff32c15ec60 msgr2=0x7ff32c0ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.635+0000 7ff330ac8640 1 --2- 172.21.15.97:0/3141218509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff32c15ec60 0x7ff32c0ce280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff32c071390 tx=0x7ff31402b840 comp rx=0 tx=0).stop 2024-03-29T04:08:04.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.635+0000 7ff330ac8640 1 -- 172.21.15.97:0/3141218509 shutdown_connections 2024-03-29T04:08:04.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.635+0000 7ff330ac8640 1 --2- 172.21.15.97:0/3141218509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff32c15ec60 0x7ff32c0ce280 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.635+0000 7ff330ac8640 1 --2- 172.21.15.97:0/3141218509 >> v2:172.21.15.154:3300/0 conn(0x7ff32c15e880 0x7ff32c0cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.635+0000 7ff330ac8640 1 -- 172.21.15.97:0/3141218509 >> 172.21.15.97:0/3141218509 conn(0x7ff32c0b6860 msgr2=0x7ff32c0b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:04.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.635+0000 7ff330ac8640 1 -- 172.21.15.97:0/3141218509 shutdown_connections 2024-03-29T04:08:04.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.636+0000 7ff330ac8640 1 -- 172.21.15.97:0/3141218509 wait complete. 2024-03-29T04:08:04.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.637+0000 7ff330ac8640 1 Processor -- start 2024-03-29T04:08:04.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.637+0000 7ff330ac8640 1 -- start start 2024-03-29T04:08:04.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff330ac8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff32c0b4340 0x7ff32c150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff330ac8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff32c15e880 0x7ff32c0ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff330ac8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff32c15ec60 0x7ff32c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff330ac8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c06bc20 con 0x7ff32c15ec60 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff330ac8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c0cfa90 con 0x7ff32c0b4340 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff330ac8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff32c06bda0 con 0x7ff32c15e880 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff32ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff32c15ec60 0x7ff32c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff32ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff32c15ec60 0x7ff32c0cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39488/0 (socket says 172.21.15.97:39488) 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff32ad9d640 1 -- 172.21.15.97:0/1048534075 learned_addr learned my addr 172.21.15.97:0/1048534075 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff32a59c640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff32c0b4340 0x7ff32c150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff323fff640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff32c15e880 0x7ff32c0ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff32ad9d640 1 -- 172.21.15.97:0/1048534075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff32c15e880 msgr2=0x7ff32c0ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff32ad9d640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff32c15e880 0x7ff32c0ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff32ad9d640 1 -- 172.21.15.97:0/1048534075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff32c0b4340 msgr2=0x7ff32c150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff32ad9d640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff32c0b4340 0x7ff32c150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff32ad9d640 1 -- 172.21.15.97:0/1048534075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32c15c4f0 con 0x7ff32c15ec60 2024-03-29T04:08:04.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff32ad9d640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff32c15ec60 0x7ff32c0cb040 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff32400c7d0 tx=0x7ff32400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:04.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff322ffd640 1 -- 172.21.15.97:0/1048534075 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324018070 con 0x7ff32c15ec60 2024-03-29T04:08:04.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff322ffd640 1 -- 172.21.15.97:0/1048534075 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff32400a640 con 0x7ff32c15ec60 2024-03-29T04:08:04.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff322ffd640 1 -- 172.21.15.97:0/1048534075 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32400aa70 con 0x7ff32c15ec60 2024-03-29T04:08:04.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff330ac8640 1 -- 172.21.15.97:0/1048534075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32c160d10 con 0x7ff32c15ec60 2024-03-29T04:08:04.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.638+0000 7ff330ac8640 1 -- 172.21.15.97:0/1048534075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff32c1612d0 con 0x7ff32c15ec60 2024-03-29T04:08:04.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.639+0000 7ff330ac8640 1 -- 172.21.15.97:0/1048534075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff32c068640 con 0x7ff32c15ec60 2024-03-29T04:08:04.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.640+0000 7ff322ffd640 1 -- 172.21.15.97:0/1048534075 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff32401b070 con 0x7ff32c15ec60 2024-03-29T04:08:04.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.640+0000 7ff322ffd640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff30c06c560 0x7ff30c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.640+0000 7ff32a59c640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff30c06c560 0x7ff30c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.640+0000 7ff322ffd640 1 -- 172.21.15.97:0/1048534075 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff32408ab30 con 0x7ff32c15ec60 2024-03-29T04:08:04.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.640+0000 7ff32a59c640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff30c06c560 0x7ff30c06ea20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009b40 tx=0x7ff31c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:04.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.645+0000 7ff322ffd640 1 -- 172.21.15.97:0/1048534075 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff324058b90 con 0x7ff32c15ec60 2024-03-29T04:08:04.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.821+0000 7ff330ac8640 1 -- 172.21.15.97:0/1048534075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff32c06ec10 con 0x7ff32c15ec60 2024-03-29T04:08:04.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.822+0000 7ff322ffd640 1 -- 172.21.15.97:0/1048534075 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7ff32405ce90 con 0x7ff32c15ec60 2024-03-29T04:08:04.802 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:04.802 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.824+0000 7ff330ac8640 1 -- 172.21.15.97:0/1048534075 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff30c06c560 msgr2=0x7ff30c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.824+0000 7ff330ac8640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff30c06c560 0x7ff30c06ea20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009b40 tx=0x7ff31c009190 comp rx=0 tx=0).stop 2024-03-29T04:08:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.824+0000 7ff330ac8640 1 -- 172.21.15.97:0/1048534075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff32c15ec60 msgr2=0x7ff32c0cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.824+0000 7ff330ac8640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff32c15ec60 0x7ff32c0cb040 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff32400c7d0 tx=0x7ff32400cc90 comp rx=0 tx=0).stop 2024-03-29T04:08:04.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.824+0000 7ff330ac8640 1 -- 172.21.15.97:0/1048534075 shutdown_connections 2024-03-29T04:08:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.824+0000 7ff330ac8640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff30c06c560 0x7ff30c06ea20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.824+0000 7ff330ac8640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff32c15ec60 0x7ff32c0cb040 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.824+0000 7ff330ac8640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff32c15e880 0x7ff32c0ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.824+0000 7ff330ac8640 1 --2- 172.21.15.97:0/1048534075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff32c0b4340 0x7ff32c150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.824+0000 7ff330ac8640 1 -- 172.21.15.97:0/1048534075 >> 172.21.15.97:0/1048534075 conn(0x7ff32c0b6860 msgr2=0x7ff32c0b5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.824+0000 7ff330ac8640 1 -- 172.21.15.97:0/1048534075 shutdown_connections 2024-03-29T04:08:04.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.824+0000 7ff330ac8640 1 -- 172.21.15.97:0/1048534075 wait complete. 2024-03-29T04:08:04.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:07:38.618704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:07:39.724339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:08:04.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:08:04.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.939+0000 7fdff45d4640 1 Processor -- start 2024-03-29T04:08:04.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.939+0000 7fdff45d4640 1 -- start start 2024-03-29T04:08:04.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.940+0000 7fdff45d4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdfec145660 0x7fdfec160220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.940+0000 7fdff45d4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfec0cf3e0 con 0x7fdfec0cfba0 2024-03-29T04:08:04.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.940+0000 7fdff45d4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfec145a40 con 0x7fdfec070110 2024-03-29T04:08:04.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.940+0000 7fdff45d4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fdfec070570 con 0x7fdfec145660 2024-03-29T04:08:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.940+0000 7fdff2370640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdfec145660 0x7fdfec160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.940+0000 7fdff2b71640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdfec0cfba0 0x7fdfec071bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33504/0 (socket says 172.21.15.97:33504) 2024-03-29T04:08:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.940+0000 7fdff2b71640 1 -- 172.21.15.97:0/88218770 learned_addr learned my addr 172.21.15.97:0/88218770 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.940+0000 7fdff2370640 1 -- 172.21.15.97:0/88218770 >> v1:172.21.15.154:6789/0 conn(0x7fdfec070110 legacy=0x7fdfec0b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.940+0000 7fdff2370640 1 -- 172.21.15.97:0/88218770 >> v1:172.21.15.97:6789/0 conn(0x7fdfec0cfba0 legacy=0x7fdfec071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.940+0000 7fdff2370640 1 -- 172.21.15.97:0/88218770 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec160920 con 0x7fdfec145660 2024-03-29T04:08:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.940+0000 7fdff136e640 1 -- 172.21.15.97:0/88218770 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2551240398 0 0) 0x7fdfec0cf3e0 con 0x7fdfec0cfba0 2024-03-29T04:08:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.941+0000 7fdff2370640 1 --2- 172.21.15.97:0/88218770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfec145660 0x7fdfec160220 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80097a0 tx=0x7fdfe802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f284eb021ec78ba server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.941+0000 7fdff136e640 1 -- 172.21.15.97:0/88218770 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8034070 con 0x7fdfec145660 2024-03-29T04:08:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.941+0000 7fdff136e640 1 -- 172.21.15.97:0/88218770 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe8030610 con 0x7fdfec145660 2024-03-29T04:08:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.941+0000 7fdff136e640 1 -- 172.21.15.97:0/88218770 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80309e0 con 0x7fdfec145660 2024-03-29T04:08:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.942+0000 7fdff45d4640 1 -- 172.21.15.97:0/88218770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfec145660 msgr2=0x7fdfec160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.942+0000 7fdff45d4640 1 --2- 172.21.15.97:0/88218770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfec145660 0x7fdfec160220 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80097a0 tx=0x7fdfe802b960 comp rx=0 tx=0).stop 2024-03-29T04:08:04.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.942+0000 7fdff45d4640 1 -- 172.21.15.97:0/88218770 shutdown_connections 2024-03-29T04:08:04.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.942+0000 7fdff45d4640 1 --2- 172.21.15.97:0/88218770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfec145660 0x7fdfec160220 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.942+0000 7fdff45d4640 1 -- 172.21.15.97:0/88218770 >> 172.21.15.97:0/88218770 conn(0x7fdfec0b2340 msgr2=0x7fdfec0b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:04.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.942+0000 7fdff45d4640 1 -- 172.21.15.97:0/88218770 shutdown_connections 2024-03-29T04:08:04.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.942+0000 7fdff45d4640 1 -- 172.21.15.97:0/88218770 wait complete. 2024-03-29T04:08:04.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.943+0000 7fdff45d4640 1 Processor -- start 2024-03-29T04:08:04.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.943+0000 7fdff45d4640 1 -- start start 2024-03-29T04:08:04.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff45d4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfec070110 0x7fdfec1f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff45d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfec0cfba0 0x7fdfec0a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff45d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfec145660 0x7fdfec143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff45d4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec0cf680 con 0x7fdfec070110 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff45d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec160ee0 con 0x7fdfec0cfba0 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff45d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdfec0b3d50 con 0x7fdfec145660 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff2370640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfec070110 0x7fdfec1f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff2370640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfec070110 0x7fdfec1f23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39512/0 (socket says 172.21.15.97:39512) 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff2370640 1 -- 172.21.15.97:0/1279615924 learned_addr learned my addr 172.21.15.97:0/1279615924 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff1b6f640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfec0cfba0 0x7fdfec0a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff2b71640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfec145660 0x7fdfec143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff2370640 1 -- 172.21.15.97:0/1279615924 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfec145660 msgr2=0x7fdfec143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff2370640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfec145660 0x7fdfec143f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff2370640 1 -- 172.21.15.97:0/1279615924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfec0cfba0 msgr2=0x7fdfec0a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:04.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff2370640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfec0cfba0 0x7fdfec0a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:04.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff2370640 1 -- 172.21.15.97:0/1279615924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec141840 con 0x7fdfec070110 2024-03-29T04:08:04.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff1b6f640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfec0cfba0 0x7fdfec0a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:04.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff2370640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfec070110 0x7fdfec1f23c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80098c0 tx=0x7fdfe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:04.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdfe2ffd640 1 -- 172.21.15.97:0/1279615924 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8034040 con 0x7fdfec070110 2024-03-29T04:08:04.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdfe2ffd640 1 -- 172.21.15.97:0/1279615924 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe8039a10 con 0x7fdfec070110 2024-03-29T04:08:04.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff45d4640 1 -- 172.21.15.97:0/1279615924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfec141b20 con 0x7fdfec070110 2024-03-29T04:08:04.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdfe2ffd640 1 -- 172.21.15.97:0/1279615924 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8039de0 con 0x7fdfec070110 2024-03-29T04:08:04.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.944+0000 7fdff45d4640 1 -- 172.21.15.97:0/1279615924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfec1420e0 con 0x7fdfec070110 2024-03-29T04:08:04.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.945+0000 7fdff45d4640 1 -- 172.21.15.97:0/1279615924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfec073580 con 0x7fdfec070110 2024-03-29T04:08:04.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.946+0000 7fdfe2ffd640 1 -- 172.21.15.97:0/1279615924 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdfe8037070 con 0x7fdfec070110 2024-03-29T04:08:04.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.946+0000 7fdfe2ffd640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfd006c530 0x7fdfd006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:04.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.946+0000 7fdff1b6f640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfd006c530 0x7fdfd006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:04.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.946+0000 7fdfe2ffd640 1 -- 172.21.15.97:0/1279615924 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdfe80ad870 con 0x7fdfec070110 2024-03-29T04:08:04.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.947+0000 7fdff1b6f640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfd006c530 0x7fdfd006e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc007fd0 tx=0x7fdfdc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:04.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:04.950+0000 7fdfe2ffd640 1 -- 172.21.15.97:0/1279615924 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdfe807be90 con 0x7fdfec070110 2024-03-29T04:08:05.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.122+0000 7fdff45d4640 1 -- 172.21.15.97:0/1279615924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdfec0afe50 con 0x7fdfec070110 2024-03-29T04:08:05.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.122+0000 7fdfe2ffd640 1 -- 172.21.15.97:0/1279615924 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fdfe807b790 con 0x7fdfec070110 2024-03-29T04:08:05.102 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:08:05.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.124+0000 7fdff45d4640 1 -- 172.21.15.97:0/1279615924 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfd006c530 msgr2=0x7fdfd006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.124+0000 7fdff45d4640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfd006c530 0x7fdfd006e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc007fd0 tx=0x7fdfdc009040 comp rx=0 tx=0).stop 2024-03-29T04:08:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.124+0000 7fdff45d4640 1 -- 172.21.15.97:0/1279615924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfec070110 msgr2=0x7fdfec1f23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.124+0000 7fdff45d4640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfec070110 0x7fdfec1f23c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80098c0 tx=0x7fdfe80046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.124+0000 7fdff45d4640 1 -- 172.21.15.97:0/1279615924 shutdown_connections 2024-03-29T04:08:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.124+0000 7fdff45d4640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfd006c530 0x7fdfd006e9f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.124+0000 7fdff45d4640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfec145660 0x7fdfec143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.124+0000 7fdff45d4640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfec0cfba0 0x7fdfec0a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.124+0000 7fdff45d4640 1 --2- 172.21.15.97:0/1279615924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfec070110 0x7fdfec1f23c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.124+0000 7fdff45d4640 1 -- 172.21.15.97:0/1279615924 >> 172.21.15.97:0/1279615924 conn(0x7fdfec0b2340 msgr2=0x7fdfec0b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.124+0000 7fdff45d4640 1 -- 172.21.15.97:0/1279615924 shutdown_connections 2024-03-29T04:08:05.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.124+0000 7fdff45d4640 1 -- 172.21.15.97:0/1279615924 wait complete. 2024-03-29T04:08:05.115 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:08:05.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.239+0000 7eff59433640 1 Processor -- start 2024-03-29T04:08:05.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.239+0000 7eff59433640 1 -- start start 2024-03-29T04:08:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.239+0000 7eff59433640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7eff540cfde0 0x7eff540d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.239+0000 7eff59433640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5415ed00 con 0x7eff5415e920 2024-03-29T04:08:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.239+0000 7eff59433640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff540d21e0 con 0x7eff540d0f90 2024-03-29T04:08:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.239+0000 7eff59433640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7eff540d2480 con 0x7eff540cfde0 2024-03-29T04:08:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.240+0000 7eff52ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7eff5415e920 0x7eff540d3240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33508/0 (socket says 172.21.15.97:33508) 2024-03-29T04:08:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.240+0000 7eff52ffd640 1 -- 172.21.15.97:0/2465640666 learned_addr learned my addr 172.21.15.97:0/2465640666 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.240+0000 7eff527fc640 1 --2- 172.21.15.97:0/2465640666 >> v2:172.21.15.154:3300/0 conn(0x7eff540cfde0 0x7eff540d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.240+0000 7eff51ffb640 1 -- 172.21.15.97:0/2465640666 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2762617855 0 0) 0x7eff5415ed00 con 0x7eff5415e920 2024-03-29T04:08:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.240+0000 7eff51ffb640 1 -- 172.21.15.97:0/2465640666 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff3c003520 con 0x7eff5415e920 2024-03-29T04:08:05.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.240+0000 7eff51ffb640 1 -- 172.21.15.97:0/2465640666 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2010777900 0 0) 0x7eff540d21e0 con 0x7eff540d0f90 2024-03-29T04:08:05.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.240+0000 7eff51ffb640 1 -- 172.21.15.97:0/2465640666 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff5415ed00 con 0x7eff540d0f90 2024-03-29T04:08:05.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.240+0000 7eff527fc640 1 -- 172.21.15.97:0/2465640666 >> v1:172.21.15.154:6789/0 conn(0x7eff540d0f90 legacy=0x7eff5406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.240+0000 7eff527fc640 1 -- 172.21.15.97:0/2465640666 >> v1:172.21.15.97:6789/0 conn(0x7eff5415e920 legacy=0x7eff540d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.240+0000 7eff527fc640 1 -- 172.21.15.97:0/2465640666 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff540d2760 con 0x7eff540cfde0 2024-03-29T04:08:05.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.241+0000 7eff527fc640 1 --2- 172.21.15.97:0/2465640666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff540cfde0 0x7eff540d0240 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff480097a0 tx=0x7eff4802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c66a7993d716440 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:05.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.241+0000 7eff51ffb640 1 -- 172.21.15.97:0/2465640666 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48034070 con 0x7eff540cfde0 2024-03-29T04:08:05.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.241+0000 7eff51ffb640 1 -- 172.21.15.97:0/2465640666 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff48030610 con 0x7eff540cfde0 2024-03-29T04:08:05.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.241+0000 7eff51ffb640 1 -- 172.21.15.97:0/2465640666 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff480309e0 con 0x7eff540cfde0 2024-03-29T04:08:05.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.242+0000 7eff59433640 1 -- 172.21.15.97:0/2465640666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff540cfde0 msgr2=0x7eff540d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.242+0000 7eff59433640 1 --2- 172.21.15.97:0/2465640666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff540cfde0 0x7eff540d0240 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff480097a0 tx=0x7eff4802b960 comp rx=0 tx=0).stop 2024-03-29T04:08:05.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.242+0000 7eff59433640 1 -- 172.21.15.97:0/2465640666 shutdown_connections 2024-03-29T04:08:05.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.242+0000 7eff59433640 1 --2- 172.21.15.97:0/2465640666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff540cfde0 0x7eff540d0240 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.242+0000 7eff59433640 1 -- 172.21.15.97:0/2465640666 >> 172.21.15.97:0/2465640666 conn(0x7eff5408bfc0 msgr2=0x7eff540d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:05.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.242+0000 7eff59433640 1 -- 172.21.15.97:0/2465640666 shutdown_connections 2024-03-29T04:08:05.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.242+0000 7eff59433640 1 -- 172.21.15.97:0/2465640666 wait complete. 2024-03-29T04:08:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.243+0000 7eff59433640 1 Processor -- start 2024-03-29T04:08:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.243+0000 7eff59433640 1 -- start start 2024-03-29T04:08:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff59433640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff540cfde0 0x7eff54160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff59433640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff540d0f90 0x7eff54159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff59433640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff5415e920 0x7eff541599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff59433640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff540d4ef0 con 0x7eff540d0f90 2024-03-29T04:08:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff59433640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7eff540d2d20 con 0x7eff540cfde0 2024-03-29T04:08:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff59433640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7eff540d5070 con 0x7eff5415e920 2024-03-29T04:08:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff527fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff540d0f90 0x7eff54159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff527fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff540d0f90 0x7eff54159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39528/0 (socket says 172.21.15.97:39528) 2024-03-29T04:08:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff527fc640 1 -- 172.21.15.97:0/631474199 learned_addr learned my addr 172.21.15.97:0/631474199 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff52ffd640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff540cfde0 0x7eff54160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff537fe640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff5415e920 0x7eff541599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff527fc640 1 -- 172.21.15.97:0/631474199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff5415e920 msgr2=0x7eff541599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff527fc640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff5415e920 0x7eff541599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff527fc640 1 -- 172.21.15.97:0/631474199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff540cfde0 msgr2=0x7eff54160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff527fc640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff540cfde0 0x7eff54160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff527fc640 1 -- 172.21.15.97:0/631474199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5414e770 con 0x7eff540d0f90 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff537fe640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff5415e920 0x7eff541599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff527fc640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff540d0f90 0x7eff54159460 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7eff48009890 tx=0x7eff480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff3b7fe640 1 -- 172.21.15.97:0/631474199 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48034040 con 0x7eff540d0f90 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff3b7fe640 1 -- 172.21.15.97:0/631474199 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff48030410 con 0x7eff540d0f90 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff3b7fe640 1 -- 172.21.15.97:0/631474199 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48006e10 con 0x7eff540d0f90 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff59433640 1 -- 172.21.15.97:0/631474199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5414ea50 con 0x7eff540d0f90 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.244+0000 7eff59433640 1 -- 172.21.15.97:0/631474199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5414f010 con 0x7eff540d0f90 2024-03-29T04:08:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.245+0000 7eff59433640 1 -- 172.21.15.97:0/631474199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff540a2560 con 0x7eff540d0f90 2024-03-29T04:08:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.246+0000 7eff3b7fe640 1 -- 172.21.15.97:0/631474199 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7eff48037070 con 0x7eff540d0f90 2024-03-29T04:08:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.246+0000 7eff3b7fe640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7eff4006c530 0x7eff4006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.246+0000 7eff3b7fe640 1 -- 172.21.15.97:0/631474199 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7eff480ad490 con 0x7eff540d0f90 2024-03-29T04:08:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.246+0000 7eff52ffd640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7eff4006c530 0x7eff4006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.247+0000 7eff52ffd640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7eff4006c530 0x7eff4006e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff44000f60 tx=0x7eff44007d70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:05.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.250+0000 7eff3b7fe640 1 -- 172.21.15.97:0/631474199 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7eff4807bb40 con 0x7eff540d0f90 2024-03-29T04:08:05.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.425+0000 7eff59433640 1 -- 172.21.15.97:0/631474199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7eff5414f1e0 con 0x7eff540d0f90 2024-03-29T04:08:05.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.425+0000 7eff3b7fe640 1 -- 172.21.15.97:0/631474199 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7eff4807b440 con 0x7eff540d0f90 2024-03-29T04:08:05.405 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:08:05.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.427+0000 7eff59433640 1 -- 172.21.15.97:0/631474199 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7eff4006c530 msgr2=0x7eff4006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.427+0000 7eff59433640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7eff4006c530 0x7eff4006e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff44000f60 tx=0x7eff44007d70 comp rx=0 tx=0).stop 2024-03-29T04:08:05.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.427+0000 7eff59433640 1 -- 172.21.15.97:0/631474199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff540d0f90 msgr2=0x7eff54159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.427+0000 7eff59433640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff540d0f90 0x7eff54159460 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7eff48009890 tx=0x7eff480046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.427+0000 7eff59433640 1 -- 172.21.15.97:0/631474199 shutdown_connections 2024-03-29T04:08:05.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.427+0000 7eff59433640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7eff4006c530 0x7eff4006e9f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.427+0000 7eff59433640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7eff5415e920 0x7eff541599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.427+0000 7eff59433640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff540d0f90 0x7eff54159460 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.427+0000 7eff59433640 1 --2- 172.21.15.97:0/631474199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff540cfde0 0x7eff54160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.427+0000 7eff59433640 1 -- 172.21.15.97:0/631474199 >> 172.21.15.97:0/631474199 conn(0x7eff5408bfc0 msgr2=0x7eff540b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:05.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.427+0000 7eff59433640 1 -- 172.21.15.97:0/631474199 shutdown_connections 2024-03-29T04:08:05.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.427+0000 7eff59433640 1 -- 172.21.15.97:0/631474199 wait complete. 2024-03-29T04:08:05.418 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:08:05.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.543+0000 7f65590d3640 1 Processor -- start 2024-03-29T04:08:05.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.543+0000 7f65590d3640 1 -- start start 2024-03-29T04:08:05.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.544+0000 7f65590d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f655409b140 0x7f6554167870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.544+0000 7f65590d3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65540cde40 con 0x7f65540ce680 2024-03-29T04:08:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.544+0000 7f65590d3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65540d0c80 con 0x7f65540cea60 2024-03-29T04:08:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.544+0000 7f65590d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f65540d0a30 con 0x7f655409b140 2024-03-29T04:08:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.544+0000 7f6552d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f65540ce680 0x7f655409b770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33522/0 (socket says 172.21.15.97:33522) 2024-03-29T04:08:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.544+0000 7f6552d9d640 1 -- 172.21.15.97:0/460123175 learned_addr learned my addr 172.21.15.97:0/460123175 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.544+0000 7f655359e640 1 --2- 172.21.15.97:0/460123175 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f655409b140 0x7f6554167870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.544+0000 7f6551d9b640 1 -- 172.21.15.97:0/460123175 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 915049191 0 0) 0x7f65540cde40 con 0x7f65540ce680 2024-03-29T04:08:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.544+0000 7f6551d9b640 1 -- 172.21.15.97:0/460123175 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f653c003520 con 0x7f65540ce680 2024-03-29T04:08:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.545+0000 7f6551d9b640 1 -- 172.21.15.97:0/460123175 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 573441941 0 0) 0x7f65540d0c80 con 0x7f65540cea60 2024-03-29T04:08:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.545+0000 7f6551d9b640 1 -- 172.21.15.97:0/460123175 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65540cde40 con 0x7f65540cea60 2024-03-29T04:08:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.545+0000 7f655359e640 1 -- 172.21.15.97:0/460123175 >> v1:172.21.15.154:6789/0 conn(0x7f65540cea60 legacy=0x7f655409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.545+0000 7f655359e640 1 -- 172.21.15.97:0/460123175 >> v1:172.21.15.97:6789/0 conn(0x7f65540ce680 legacy=0x7f655409b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.545+0000 7f655359e640 1 -- 172.21.15.97:0/460123175 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65540d0f60 con 0x7f655409b140 2024-03-29T04:08:05.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.545+0000 7f655359e640 1 --2- 172.21.15.97:0/460123175 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f655409b140 0x7f6554167870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f654c0097d0 tx=0x7f654c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5582186aca1b2e3d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:05.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.546+0000 7f6551d9b640 1 -- 172.21.15.97:0/460123175 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c035020 con 0x7f655409b140 2024-03-29T04:08:05.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.546+0000 7f6551d9b640 1 -- 172.21.15.97:0/460123175 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f654c031800 con 0x7f655409b140 2024-03-29T04:08:05.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.546+0000 7f6551d9b640 1 -- 172.21.15.97:0/460123175 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c031bb0 con 0x7f655409b140 2024-03-29T04:08:05.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.546+0000 7f65590d3640 1 -- 172.21.15.97:0/460123175 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f655409b140 msgr2=0x7f6554167870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.546+0000 7f65590d3640 1 --2- 172.21.15.97:0/460123175 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f655409b140 0x7f6554167870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f654c0097d0 tx=0x7f654c02b8a0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.546+0000 7f65590d3640 1 -- 172.21.15.97:0/460123175 shutdown_connections 2024-03-29T04:08:05.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.546+0000 7f65590d3640 1 --2- 172.21.15.97:0/460123175 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f655409b140 0x7f6554167870 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.546+0000 7f65590d3640 1 -- 172.21.15.97:0/460123175 >> 172.21.15.97:0/460123175 conn(0x7f65540b3f40 msgr2=0x7f65540b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:05.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.546+0000 7f65590d3640 1 -- 172.21.15.97:0/460123175 shutdown_connections 2024-03-29T04:08:05.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.546+0000 7f65590d3640 1 -- 172.21.15.97:0/460123175 wait complete. 2024-03-29T04:08:05.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.548+0000 7f65590d3640 1 Processor -- start 2024-03-29T04:08:05.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.548+0000 7f65590d3640 1 -- start start 2024-03-29T04:08:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.548+0000 7f65590d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f655409b140 0x7f65540b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.548+0000 7f65590d3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65540ce680 0x7f65541f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.548+0000 7f65590d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cea60 0x7f65541f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.548+0000 7f65590d3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6554070ae0 con 0x7f65540ce680 2024-03-29T04:08:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.548+0000 7f65590d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6554070650 con 0x7f65540cea60 2024-03-29T04:08:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.548+0000 7f65590d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6554070c60 con 0x7f655409b140 2024-03-29T04:08:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.548+0000 7f655259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65540ce680 0x7f65541f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.548+0000 7f655259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65540ce680 0x7f65541f37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39534/0 (socket says 172.21.15.97:39534) 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.548+0000 7f655259c640 1 -- 172.21.15.97:0/2351126857 learned_addr learned my addr 172.21.15.97:0/2351126857 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.548+0000 7f6552d9d640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f655409b140 0x7f65540b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.548+0000 7f655359e640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cea60 0x7f65541f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.549+0000 7f655259c640 1 -- 172.21.15.97:0/2351126857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f655409b140 msgr2=0x7f65540b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.549+0000 7f655259c640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f655409b140 0x7f65540b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.549+0000 7f655259c640 1 -- 172.21.15.97:0/2351126857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cea60 msgr2=0x7f65541f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.549+0000 7f655259c640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cea60 0x7f65541f3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.549+0000 7f655259c640 1 -- 172.21.15.97:0/2351126857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65540b6840 con 0x7f65540ce680 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.549+0000 7f6552d9d640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f655409b140 0x7f65540b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.549+0000 7f655359e640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cea60 0x7f65541f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.549+0000 7f655259c640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65540ce680 0x7f65541f37e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f654800c430 tx=0x7f654800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.549+0000 7f653b7fe640 1 -- 172.21.15.97:0/2351126857 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548017070 con 0x7f65540ce680 2024-03-29T04:08:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.549+0000 7f653b7fe640 1 -- 172.21.15.97:0/2351126857 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65480052d0 con 0x7f65540ce680 2024-03-29T04:08:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.549+0000 7f653b7fe640 1 -- 172.21.15.97:0/2351126857 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548003810 con 0x7f65540ce680 2024-03-29T04:08:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.549+0000 7f65590d3640 1 -- 172.21.15.97:0/2351126857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65540b6b20 con 0x7f65540ce680 2024-03-29T04:08:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.549+0000 7f65590d3640 1 -- 172.21.15.97:0/2351126857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65540b70e0 con 0x7f65540ce680 2024-03-29T04:08:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.550+0000 7f65590d3640 1 -- 172.21.15.97:0/2351126857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65540a0520 con 0x7f65540ce680 2024-03-29T04:08:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.550+0000 7f653b7fe640 1 -- 172.21.15.97:0/2351126857 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f654801a070 con 0x7f65540ce680 2024-03-29T04:08:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.551+0000 7f653b7fe640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f654006c530 0x7f654006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.551+0000 7f653b7fe640 1 -- 172.21.15.97:0/2351126857 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f654808aa70 con 0x7f65540ce680 2024-03-29T04:08:05.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.551+0000 7f6552d9d640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f654006c530 0x7f654006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.551+0000 7f6552d9d640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f654006c530 0x7f654006e9f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f65440012e0 tx=0x7f65440019f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:05.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.555+0000 7f653b7fe640 1 -- 172.21.15.97:0/2351126857 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6548058a40 con 0x7f65540ce680 2024-03-29T04:08:05.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.729+0000 7f65590d3640 1 -- 172.21.15.97:0/2351126857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f65540af3d0 con 0x7f65540ce680 2024-03-29T04:08:05.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.730+0000 7f653b7fe640 1 -- 172.21.15.97:0/2351126857 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f654805cd40 con 0x7f65540ce680 2024-03-29T04:08:05.709 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:08:05.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.732+0000 7f65590d3640 1 -- 172.21.15.97:0/2351126857 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f654006c530 msgr2=0x7f654006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.732+0000 7f65590d3640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f654006c530 0x7f654006e9f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f65440012e0 tx=0x7f65440019f0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.732+0000 7f65590d3640 1 -- 172.21.15.97:0/2351126857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65540ce680 msgr2=0x7f65541f37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.732+0000 7f65590d3640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65540ce680 0x7f65541f37e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f654800c430 tx=0x7f654800c8f0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.732+0000 7f65590d3640 1 -- 172.21.15.97:0/2351126857 shutdown_connections 2024-03-29T04:08:05.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.732+0000 7f65590d3640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f654006c530 0x7f654006e9f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.732+0000 7f65590d3640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cea60 0x7f65541f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.732+0000 7f65590d3640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65540ce680 0x7f65541f37e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.732+0000 7f65590d3640 1 --2- 172.21.15.97:0/2351126857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f655409b140 0x7f65540b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.732+0000 7f65590d3640 1 -- 172.21.15.97:0/2351126857 >> 172.21.15.97:0/2351126857 conn(0x7f65540b3f40 msgr2=0x7f65540b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:05.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.732+0000 7f65590d3640 1 -- 172.21.15.97:0/2351126857 shutdown_connections 2024-03-29T04:08:05.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.732+0000 7f65590d3640 1 -- 172.21.15.97:0/2351126857 wait complete. 2024-03-29T04:08:05.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:05.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.845+0000 7f6a8730e640 1 Processor -- start 2024-03-29T04:08:05.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.845+0000 7f6a8730e640 1 -- start start 2024-03-29T04:08:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a8730e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6a801481f0 0x7f6a800b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a8730e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6a8006f440 0x7f6a800b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a8730e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a80071c40 0x7f6a80072060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a8730e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f6a800ad9e0 con 0x7f6a8006f440 2024-03-29T04:08:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a8730e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f6a8006e740 con 0x7f6a801481f0 2024-03-29T04:08:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a8730e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6a8006ea20 con 0x7f6a80071c40 2024-03-29T04:08:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a848a9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6a8006f440 0x7f6a800b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a848a9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6a8006f440 0x7f6a800b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39546/0 (socket says 172.21.15.97:39546) 2024-03-29T04:08:05.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a848a9640 1 -- 172.21.15.97:0/2880239019 learned_addr learned my addr 172.21.15.97:0/2880239019 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:05.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a850aa640 1 --2- 172.21.15.97:0/2880239019 >> v2:172.21.15.154:3300/0 conn(0x7f6a801481f0 0x7f6a800b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a858ab640 1 --2- 172.21.15.97:0/2880239019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a80071c40 0x7f6a80072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a848a9640 1 -- 172.21.15.97:0/2880239019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a80071c40 msgr2=0x7f6a80072060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:05.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a848a9640 1 --2- 172.21.15.97:0/2880239019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a80071c40 0x7f6a80072060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a848a9640 1 -- 172.21.15.97:0/2880239019 >> v2:172.21.15.154:3300/0 conn(0x7f6a801481f0 msgr2=0x7f6a800b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:05.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a848a9640 1 --2- 172.21.15.97:0/2880239019 >> v2:172.21.15.154:3300/0 conn(0x7f6a801481f0 0x7f6a800b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.846+0000 7f6a848a9640 1 -- 172.21.15.97:0/2880239019 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8006ed00 con 0x7f6a8006f440 2024-03-29T04:08:05.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.847+0000 7f6a848a9640 1 --2- 172.21.15.97:0/2880239019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8006f440 0x7f6a800b44a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6a80075190 tx=0x7f6a7002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f8ac63b81cd26d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:05.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.847+0000 7f6a77fff640 1 -- 172.21.15.97:0/2880239019 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70035020 con 0x7f6a8006f440 2024-03-29T04:08:05.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.847+0000 7f6a77fff640 1 -- 172.21.15.97:0/2880239019 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a70031870 con 0x7f6a8006f440 2024-03-29T04:08:05.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.847+0000 7f6a77fff640 1 -- 172.21.15.97:0/2880239019 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70031c40 con 0x7f6a8006f440 2024-03-29T04:08:05.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.848+0000 7f6a8730e640 1 -- 172.21.15.97:0/2880239019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8006f440 msgr2=0x7f6a800b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.848+0000 7f6a8730e640 1 --2- 172.21.15.97:0/2880239019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8006f440 0x7f6a800b44a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6a80075190 tx=0x7f6a7002b910 comp rx=0 tx=0).stop 2024-03-29T04:08:05.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.848+0000 7f6a8730e640 1 -- 172.21.15.97:0/2880239019 shutdown_connections 2024-03-29T04:08:05.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.848+0000 7f6a8730e640 1 --2- 172.21.15.97:0/2880239019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a80071c40 0x7f6a80072060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.848+0000 7f6a8730e640 1 --2- 172.21.15.97:0/2880239019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8006f440 0x7f6a800b44a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.848+0000 7f6a8730e640 1 --2- 172.21.15.97:0/2880239019 >> v2:172.21.15.154:3300/0 conn(0x7f6a801481f0 0x7f6a800b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.848+0000 7f6a8730e640 1 -- 172.21.15.97:0/2880239019 >> 172.21.15.97:0/2880239019 conn(0x7f6a8006b510 msgr2=0x7f6a8006b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.848+0000 7f6a8730e640 1 -- 172.21.15.97:0/2880239019 shutdown_connections 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.848+0000 7f6a8730e640 1 -- 172.21.15.97:0/2880239019 wait complete. 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.849+0000 7f6a8730e640 1 Processor -- start 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.849+0000 7f6a8730e640 1 -- start start 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a8730e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8006f440 0x7f6a8015e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a8730e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a80071c40 0x7f6a8013e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a8730e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a801481f0 0x7f6a80145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a8730e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a800d1480 con 0x7f6a8006f440 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a8730e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801485d0 con 0x7f6a801481f0 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a8730e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6a800d1600 con 0x7f6a80071c40 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a850aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8006f440 0x7f6a8015e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a850aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8006f440 0x7f6a8015e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39558/0 (socket says 172.21.15.97:39558) 2024-03-29T04:08:05.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a850aa640 1 -- 172.21.15.97:0/846426839 learned_addr learned my addr 172.21.15.97:0/846426839 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a848a9640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a80071c40 0x7f6a8013e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a858ab640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a801481f0 0x7f6a80145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a850aa640 1 -- 172.21.15.97:0/846426839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a80071c40 msgr2=0x7f6a8013e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a850aa640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a80071c40 0x7f6a8013e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a850aa640 1 -- 172.21.15.97:0/846426839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a801481f0 msgr2=0x7f6a80145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a850aa640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a801481f0 0x7f6a80145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a850aa640 1 -- 172.21.15.97:0/846426839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a801f52c0 con 0x7f6a8006f440 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a858ab640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a801481f0 0x7f6a80145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a850aa640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8006f440 0x7f6a8015e320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00efc0 tx=0x7f6a7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.850+0000 7f6a75ffb640 1 -- 172.21.15.97:0/846426839 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c018020 con 0x7f6a8006f440 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.851+0000 7f6a75ffb640 1 -- 172.21.15.97:0/846426839 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a7c014480 con 0x7f6a8006f440 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.851+0000 7f6a75ffb640 1 -- 172.21.15.97:0/846426839 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c014850 con 0x7f6a8006f440 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.851+0000 7f6a8730e640 1 -- 172.21.15.97:0/846426839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8013e5a0 con 0x7f6a8006f440 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.851+0000 7f6a8730e640 1 -- 172.21.15.97:0/846426839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a801f4a20 con 0x7f6a8006f440 2024-03-29T04:08:05.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.852+0000 7f6a8730e640 1 -- 172.21.15.97:0/846426839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a80068640 con 0x7f6a8006f440 2024-03-29T04:08:05.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.852+0000 7f6a75ffb640 1 -- 172.21.15.97:0/846426839 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6a7c005d10 con 0x7f6a8006f440 2024-03-29T04:08:05.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.852+0000 7f6a75ffb640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6a6406c530 0x7f6a6406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:05.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.853+0000 7f6a75ffb640 1 -- 172.21.15.97:0/846426839 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6a7c08c0f0 con 0x7f6a8006f440 2024-03-29T04:08:05.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.853+0000 7f6a848a9640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6a6406c530 0x7f6a6406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:05.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.853+0000 7f6a848a9640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6a6406c530 0x7f6a6406e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a70036000 tx=0x7f6a7003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:05.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:05.857+0000 7f6a75ffb640 1 -- 172.21.15.97:0/846426839 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6a7c022070 con 0x7f6a8006f440 2024-03-29T04:08:06.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.033+0000 7f6a8730e640 1 -- 172.21.15.97:0/846426839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a800cce30 con 0x7f6a8006f440 2024-03-29T04:08:06.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.035+0000 7f6a75ffb640 1 -- 172.21.15.97:0/846426839 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f6a7c010040 con 0x7f6a8006f440 2024-03-29T04:08:06.015 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:06.015 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.037+0000 7f6a8730e640 1 -- 172.21.15.97:0/846426839 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6a6406c530 msgr2=0x7f6a6406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.037+0000 7f6a8730e640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6a6406c530 0x7f6a6406e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a70036000 tx=0x7f6a7003c040 comp rx=0 tx=0).stop 2024-03-29T04:08:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.037+0000 7f6a8730e640 1 -- 172.21.15.97:0/846426839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8006f440 msgr2=0x7f6a8015e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.037+0000 7f6a8730e640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8006f440 0x7f6a8015e320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00efc0 tx=0x7f6a7c00c490 comp rx=0 tx=0).stop 2024-03-29T04:08:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.037+0000 7f6a8730e640 1 -- 172.21.15.97:0/846426839 shutdown_connections 2024-03-29T04:08:06.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.037+0000 7f6a8730e640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6a6406c530 0x7f6a6406e9f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.037+0000 7f6a8730e640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a801481f0 0x7f6a80145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.037+0000 7f6a8730e640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a80071c40 0x7f6a8013e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.037+0000 7f6a8730e640 1 --2- 172.21.15.97:0/846426839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8006f440 0x7f6a8015e320 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.037+0000 7f6a8730e640 1 -- 172.21.15.97:0/846426839 >> 172.21.15.97:0/846426839 conn(0x7f6a8006b510 msgr2=0x7f6a800cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:06.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.037+0000 7f6a8730e640 1 -- 172.21.15.97:0/846426839 shutdown_connections 2024-03-29T04:08:06.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.037+0000 7f6a8730e640 1 -- 172.21.15.97:0/846426839 wait complete. 2024-03-29T04:08:06.027 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:07:38.618704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:07:39.724339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:08:06.028 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:08:06.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.151+0000 7f09ac5d4640 1 Processor -- start 2024-03-29T04:08:06.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.151+0000 7f09ac5d4640 1 -- start start 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.152+0000 7f09ac5d4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09a415f6e0 0x7f09a40b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.152+0000 7f09ac5d4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f09a40b2a70 0x7f09a415fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.152+0000 7f09ac5d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09a40cf3f0 0x7f09a40afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.152+0000 7f09ac5d4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f09a4160140 con 0x7f09a40b2a70 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.152+0000 7f09ac5d4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f09a40b1b60 con 0x7f09a415f6e0 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.152+0000 7f09ac5d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f09a40b01f0 con 0x7f09a40cf3f0 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.152+0000 7f09a9b6f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f09a40b2a70 0x7f09a415fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.152+0000 7f09a9b6f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f09a40b2a70 0x7f09a415fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39562/0 (socket says 172.21.15.97:39562) 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.152+0000 7f09a9b6f640 1 -- 172.21.15.97:0/1316320023 learned_addr learned my addr 172.21.15.97:0/1316320023 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.152+0000 7f09aa370640 1 --2- 172.21.15.97:0/1316320023 >> v2:172.21.15.154:3300/0 conn(0x7f09a415f6e0 0x7f09a40b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.153+0000 7f09aab71640 1 --2- 172.21.15.97:0/1316320023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09a40cf3f0 0x7f09a40afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.153+0000 7f09a9b6f640 1 -- 172.21.15.97:0/1316320023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09a40cf3f0 msgr2=0x7f09a40afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.153+0000 7f09a9b6f640 1 --2- 172.21.15.97:0/1316320023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09a40cf3f0 0x7f09a40afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.153+0000 7f09a9b6f640 1 -- 172.21.15.97:0/1316320023 >> v2:172.21.15.154:3300/0 conn(0x7f09a415f6e0 msgr2=0x7f09a40b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:06.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.153+0000 7f09a9b6f640 1 --2- 172.21.15.97:0/1316320023 >> v2:172.21.15.154:3300/0 conn(0x7f09a415f6e0 0x7f09a40b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.153+0000 7f09a9b6f640 1 -- 172.21.15.97:0/1316320023 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a40b03a0 con 0x7f09a40b2a70 2024-03-29T04:08:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.153+0000 7f09a9b6f640 1 --2- 172.21.15.97:0/1316320023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09a40b2a70 0x7f09a415fac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f09a4072180 tx=0x7f099402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872c1386a0bdce93 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.154+0000 7f09a936e640 1 -- 172.21.15.97:0/1316320023 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099402e070 con 0x7f09a40b2a70 2024-03-29T04:08:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.154+0000 7f09a936e640 1 -- 172.21.15.97:0/1316320023 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0994037070 con 0x7f09a40b2a70 2024-03-29T04:08:06.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.154+0000 7f09a936e640 1 -- 172.21.15.97:0/1316320023 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994031a00 con 0x7f09a40b2a70 2024-03-29T04:08:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.154+0000 7f09ac5d4640 1 -- 172.21.15.97:0/1316320023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09a40b2a70 msgr2=0x7f09a415fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.154+0000 7f09ac5d4640 1 --2- 172.21.15.97:0/1316320023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09a40b2a70 0x7f09a415fac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f09a4072180 tx=0x7f099402b880 comp rx=0 tx=0).stop 2024-03-29T04:08:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.154+0000 7f09ac5d4640 1 -- 172.21.15.97:0/1316320023 shutdown_connections 2024-03-29T04:08:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.154+0000 7f09ac5d4640 1 --2- 172.21.15.97:0/1316320023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09a40cf3f0 0x7f09a40afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.154+0000 7f09ac5d4640 1 --2- 172.21.15.97:0/1316320023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09a40b2a70 0x7f09a415fac0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.154+0000 7f09ac5d4640 1 --2- 172.21.15.97:0/1316320023 >> v2:172.21.15.154:3300/0 conn(0x7f09a415f6e0 0x7f09a40b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.154+0000 7f09ac5d4640 1 -- 172.21.15.97:0/1316320023 >> 172.21.15.97:0/1316320023 conn(0x7f09a40b2ed0 msgr2=0x7f09a40b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.154+0000 7f09ac5d4640 1 -- 172.21.15.97:0/1316320023 shutdown_connections 2024-03-29T04:08:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.154+0000 7f09ac5d4640 1 -- 172.21.15.97:0/1316320023 wait complete. 2024-03-29T04:08:06.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.155+0000 7f09ac5d4640 1 Processor -- start 2024-03-29T04:08:06.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.155+0000 7f09ac5d4640 1 -- start start 2024-03-29T04:08:06.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09ac5d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09a40b2a70 0x7f09a40c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09ac5d4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09a40cf3f0 0x7f09a40c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09ac5d4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09a415f6e0 0x7f09a4150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09ac5d4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f09a406b690 con 0x7f09a415f6e0 2024-03-29T04:08:06.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09ac5d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f09a406b510 con 0x7f09a40b2a70 2024-03-29T04:08:06.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09ac5d4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f09a406b810 con 0x7f09a40cf3f0 2024-03-29T04:08:06.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09aab71640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09a415f6e0 0x7f09a4150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09aa370640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09a40b2a70 0x7f09a40c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09aab71640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09a415f6e0 0x7f09a4150590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39568/0 (socket says 172.21.15.97:39568) 2024-03-29T04:08:06.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09aab71640 1 -- 172.21.15.97:0/3295604269 learned_addr learned my addr 172.21.15.97:0/3295604269 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:06.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09a9b6f640 1 --2- 172.21.15.97:0/3295604269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09a40cf3f0 0x7f09a40c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09aab71640 1 -- 172.21.15.97:0/3295604269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09a40cf3f0 msgr2=0x7f09a40c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09aab71640 1 --2- 172.21.15.97:0/3295604269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09a40cf3f0 0x7f09a40c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09aab71640 1 -- 172.21.15.97:0/3295604269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09a40b2a70 msgr2=0x7f09a40c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.156+0000 7f09aab71640 1 --2- 172.21.15.97:0/3295604269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09a40b2a70 0x7f09a40c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.157+0000 7f09aab71640 1 -- 172.21.15.97:0/3295604269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a40c9ed0 con 0x7f09a415f6e0 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.157+0000 7f09aab71640 1 --2- 172.21.15.97:0/3295604269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09a415f6e0 0x7f09a4150590 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f099c00e860 tx=0x7f099c00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.157+0000 7f099affd640 1 -- 172.21.15.97:0/3295604269 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c017070 con 0x7f09a415f6e0 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.157+0000 7f099affd640 1 -- 172.21.15.97:0/3295604269 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f099c00c940 con 0x7f09a415f6e0 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.157+0000 7f09ac5d4640 1 -- 172.21.15.97:0/3295604269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a40c8e30 con 0x7f09a415f6e0 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.157+0000 7f099affd640 1 -- 172.21.15.97:0/3295604269 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c00cd30 con 0x7f09a415f6e0 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.157+0000 7f09ac5d4640 1 -- 172.21.15.97:0/3295604269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a40c93f0 con 0x7f09a415f6e0 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.158+0000 7f09ac5d4640 1 -- 172.21.15.97:0/3295604269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a409f530 con 0x7f09a415f6e0 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.158+0000 7f099affd640 1 -- 172.21.15.97:0/3295604269 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f099c01a070 con 0x7f09a415f6e0 2024-03-29T04:08:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.159+0000 7f099affd640 1 --2- 172.21.15.97:0/3295604269 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f098806c530 0x7f098806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.159+0000 7f099affd640 1 -- 172.21.15.97:0/3295604269 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f099c08b610 con 0x7f09a415f6e0 2024-03-29T04:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.159+0000 7f09aa370640 1 --2- 172.21.15.97:0/3295604269 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f098806c530 0x7f098806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.159+0000 7f09aa370640 1 --2- 172.21.15.97:0/3295604269 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f098806c530 0x7f098806e9f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f09a40ca5e0 tx=0x7f09a000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.163+0000 7f099affd640 1 -- 172.21.15.97:0/3295604269 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f099c0595e0 con 0x7f09a415f6e0 2024-03-29T04:08:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.339+0000 7f09ac5d4640 1 -- 172.21.15.97:0/3295604269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f09a4073400 con 0x7f09a415f6e0 2024-03-29T04:08:06.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.339+0000 7f099affd640 1 -- 172.21.15.97:0/3295604269 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f099c05d8e0 con 0x7f09a415f6e0 2024-03-29T04:08:06.319 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:08:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.341+0000 7f09ac5d4640 1 -- 172.21.15.97:0/3295604269 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f098806c530 msgr2=0x7f098806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.341+0000 7f09ac5d4640 1 --2- 172.21.15.97:0/3295604269 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f098806c530 0x7f098806e9f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f09a40ca5e0 tx=0x7f09a000b040 comp rx=0 tx=0).stop 2024-03-29T04:08:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.341+0000 7f09ac5d4640 1 -- 172.21.15.97:0/3295604269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09a415f6e0 msgr2=0x7f09a4150590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.341+0000 7f09ac5d4640 1 --2- 172.21.15.97:0/3295604269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09a415f6e0 0x7f09a4150590 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f099c00e860 tx=0x7f099c00ed20 comp rx=0 tx=0).stop 2024-03-29T04:08:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.341+0000 7f09ac5d4640 1 -- 172.21.15.97:0/3295604269 shutdown_connections 2024-03-29T04:08:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.341+0000 7f09ac5d4640 1 --2- 172.21.15.97:0/3295604269 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f098806c530 0x7f098806e9f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.341+0000 7f09ac5d4640 1 --2- 172.21.15.97:0/3295604269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09a415f6e0 0x7f09a4150590 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.341+0000 7f09ac5d4640 1 --2- 172.21.15.97:0/3295604269 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09a40cf3f0 0x7f09a40c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.341+0000 7f09ac5d4640 1 --2- 172.21.15.97:0/3295604269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09a40b2a70 0x7f09a40c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.341+0000 7f09ac5d4640 1 -- 172.21.15.97:0/3295604269 >> 172.21.15.97:0/3295604269 conn(0x7f09a40b2ed0 msgr2=0x7f09a40700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.341+0000 7f09ac5d4640 1 -- 172.21.15.97:0/3295604269 shutdown_connections 2024-03-29T04:08:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.341+0000 7f09ac5d4640 1 -- 172.21.15.97:0/3295604269 wait complete. 2024-03-29T04:08:06.331 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:08:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.455+0000 7f1b9335f640 1 Processor -- start 2024-03-29T04:08:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.456+0000 7f1b9335f640 1 -- start start 2024-03-29T04:08:06.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.456+0000 7f1b9335f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b8c0b5dd0 0x7f1b8c15fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.456+0000 7f1b9335f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1622e0 0x7f1b8c162720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.456+0000 7f1b9335f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b8c06e070 con 0x7f1b8c06f110 2024-03-29T04:08:06.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.456+0000 7f1b9335f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f1b8c06ec80 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.456+0000 7f1b9335f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8c162c60 con 0x7f1b8c1622e0 2024-03-29T04:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.456+0000 7f1b910fb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b8c0b5dd0 0x7f1b8c15fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.457+0000 7f1b910fb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b8c0b5dd0 0x7f1b8c15fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39570/0 (socket says 172.21.15.97:39570) 2024-03-29T04:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.457+0000 7f1b910fb640 1 -- 172.21.15.97:0/1958630993 learned_addr learned my addr 172.21.15.97:0/1958630993 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.457+0000 7f1b908fa640 1 --2- 172.21.15.97:0/1958630993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1622e0 0x7f1b8c162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.457+0000 7f1b910fb640 1 -- 172.21.15.97:0/1958630993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1622e0 msgr2=0x7f1b8c162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.457+0000 7f1b910fb640 1 --2- 172.21.15.97:0/1958630993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1622e0 0x7f1b8c162720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.457+0000 7f1b910fb640 1 -- 172.21.15.97:0/1958630993 >> v1:172.21.15.154:6789/0 conn(0x7f1b8c06f110 legacy=0x7f1b8c06f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.457+0000 7f1b910fb640 1 -- 172.21.15.97:0/1958630993 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8c06e350 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.458+0000 7f1b910fb640 1 --2- 172.21.15.97:0/1958630993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c0b5dd0 0x7f1b8c15fe00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009e50 tx=0x7f1b8802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daffa39f9ea39d61 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.458+0000 7f1b83fff640 1 -- 172.21.15.97:0/1958630993 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8802cb00 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.458+0000 7f1b83fff640 1 -- 172.21.15.97:0/1958630993 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8802cca0 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.458+0000 7f1b83fff640 1 -- 172.21.15.97:0/1958630993 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88034610 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.458+0000 7f1b9335f640 1 -- 172.21.15.97:0/1958630993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c0b5dd0 msgr2=0x7f1b8c15fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.458+0000 7f1b9335f640 1 --2- 172.21.15.97:0/1958630993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c0b5dd0 0x7f1b8c15fe00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009e50 tx=0x7f1b8802bf60 comp rx=0 tx=0).stop 2024-03-29T04:08:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.458+0000 7f1b9335f640 1 -- 172.21.15.97:0/1958630993 shutdown_connections 2024-03-29T04:08:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.458+0000 7f1b9335f640 1 --2- 172.21.15.97:0/1958630993 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1622e0 0x7f1b8c162720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.458+0000 7f1b9335f640 1 --2- 172.21.15.97:0/1958630993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c0b5dd0 0x7f1b8c15fe00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.458+0000 7f1b9335f640 1 -- 172.21.15.97:0/1958630993 >> 172.21.15.97:0/1958630993 conn(0x7f1b8c06b510 msgr2=0x7f1b8c06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.458+0000 7f1b9335f640 1 -- 172.21.15.97:0/1958630993 shutdown_connections 2024-03-29T04:08:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.458+0000 7f1b9335f640 1 -- 172.21.15.97:0/1958630993 wait complete. 2024-03-29T04:08:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.459+0000 7f1b9335f640 1 Processor -- start 2024-03-29T04:08:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.460+0000 7f1b9335f640 1 -- start start 2024-03-29T04:08:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.460+0000 7f1b9335f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c06f110 0x7f1b8c1f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.460+0000 7f1b9335f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c0b5dd0 0x7f1b8c1f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.460+0000 7f1b9335f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1622e0 0x7f1b8c1f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.460+0000 7f1b9335f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c160340 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.460+0000 7f1b9335f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c06e910 con 0x7f1b8c06f110 2024-03-29T04:08:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.460+0000 7f1b9335f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8c1604c0 con 0x7f1b8c1622e0 2024-03-29T04:08:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.460+0000 7f1b908fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c0b5dd0 0x7f1b8c1f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.460+0000 7f1b908fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c0b5dd0 0x7f1b8c1f82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39586/0 (socket says 172.21.15.97:39586) 2024-03-29T04:08:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.460+0000 7f1b908fa640 1 -- 172.21.15.97:0/2609803222 learned_addr learned my addr 172.21.15.97:0/2609803222 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.460+0000 7f1b910fb640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c06f110 0x7f1b8c1f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.460+0000 7f1b918fc640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1622e0 0x7f1b8c1f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.461+0000 7f1b908fa640 1 -- 172.21.15.97:0/2609803222 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1622e0 msgr2=0x7f1b8c1f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.461+0000 7f1b908fa640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1622e0 0x7f1b8c1f87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.461+0000 7f1b908fa640 1 -- 172.21.15.97:0/2609803222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c06f110 msgr2=0x7f1b8c1f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.461+0000 7f1b908fa640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c06f110 0x7f1b8c1f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.461+0000 7f1b908fa640 1 -- 172.21.15.97:0/2609803222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8c1f9110 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.461+0000 7f1b908fa640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c0b5dd0 0x7f1b8c1f82b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00e8a0 tx=0x7f1b7c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.461+0000 7f1b81ffb640 1 -- 172.21.15.97:0/2609803222 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c017070 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.461+0000 7f1b81ffb640 1 -- 172.21.15.97:0/2609803222 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b7c00c940 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.461+0000 7f1b9335f640 1 -- 172.21.15.97:0/2609803222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8c15b490 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.461+0000 7f1b81ffb640 1 -- 172.21.15.97:0/2609803222 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c00cd30 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.461+0000 7f1b9335f640 1 -- 172.21.15.97:0/2609803222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8c15ba50 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.462+0000 7f1b9335f640 1 -- 172.21.15.97:0/2609803222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b8c073a40 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.462+0000 7f1b81ffb640 1 -- 172.21.15.97:0/2609803222 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1b7c01a070 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.463+0000 7f1b81ffb640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b7006c530 0x7f1b7006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.463+0000 7f1b81ffb640 1 -- 172.21.15.97:0/2609803222 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1b7c08b440 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.463+0000 7f1b910fb640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b7006c530 0x7f1b7006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.463+0000 7f1b910fb640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b7006c530 0x7f1b7006e9f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1b88035000 tx=0x7f1b88030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:06.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.467+0000 7f1b81ffb640 1 -- 172.21.15.97:0/2609803222 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1b8c1f9110 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.643+0000 7f1b9335f640 1 -- 172.21.15.97:0/2609803222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1b8c04f9d0 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.643+0000 7f1b81ffb640 1 -- 172.21.15.97:0/2609803222 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f1b7c0594a0 con 0x7f1b8c0b5dd0 2024-03-29T04:08:06.624 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:08:06.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.645+0000 7f1b9335f640 1 -- 172.21.15.97:0/2609803222 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b7006c530 msgr2=0x7f1b7006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.645+0000 7f1b9335f640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b7006c530 0x7f1b7006e9f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1b88035000 tx=0x7f1b88030000 comp rx=0 tx=0).stop 2024-03-29T04:08:06.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.645+0000 7f1b9335f640 1 -- 172.21.15.97:0/2609803222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c0b5dd0 msgr2=0x7f1b8c1f82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.645+0000 7f1b9335f640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c0b5dd0 0x7f1b8c1f82b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00e8a0 tx=0x7f1b7c00ed60 comp rx=0 tx=0).stop 2024-03-29T04:08:06.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.645+0000 7f1b9335f640 1 -- 172.21.15.97:0/2609803222 shutdown_connections 2024-03-29T04:08:06.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.645+0000 7f1b9335f640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b7006c530 0x7f1b7006e9f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.645+0000 7f1b9335f640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b8c1622e0 0x7f1b8c1f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.645+0000 7f1b9335f640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c0b5dd0 0x7f1b8c1f82b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.645+0000 7f1b9335f640 1 --2- 172.21.15.97:0/2609803222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b8c06f110 0x7f1b8c1f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.645+0000 7f1b9335f640 1 -- 172.21.15.97:0/2609803222 >> 172.21.15.97:0/2609803222 conn(0x7f1b8c06b510 msgr2=0x7f1b8c0b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:06.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.645+0000 7f1b9335f640 1 -- 172.21.15.97:0/2609803222 shutdown_connections 2024-03-29T04:08:06.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.645+0000 7f1b9335f640 1 -- 172.21.15.97:0/2609803222 wait complete. 2024-03-29T04:08:06.636 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:08:06.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.759+0000 7fe8df58d640 1 Processor -- start 2024-03-29T04:08:06.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.759+0000 7fe8df58d640 1 -- start start 2024-03-29T04:08:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8df58d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8d815e920 0x7fe8d80d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8df58d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d80cfde0 0x7fe8d80d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8df58d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d80d3780 con 0x7fe8d80d0f90 2024-03-29T04:08:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8df58d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe8d815ed00 con 0x7fe8d815e920 2024-03-29T04:08:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8df58d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d80d21e0 con 0x7fe8d80cfde0 2024-03-29T04:08:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8dd329640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8d815e920 0x7fe8d80d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8dd329640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8d815e920 0x7fe8d80d3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39596/0 (socket says 172.21.15.97:39596) 2024-03-29T04:08:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8ddb2a640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe8d80d0f90 0x7fe8d806c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:59760/0 (socket says 172.21.15.97:59760) 2024-03-29T04:08:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8dd329640 1 -- 172.21.15.97:0/1797775002 learned_addr learned my addr 172.21.15.97:0/1797775002 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8dcb28640 1 --2- 172.21.15.97:0/1797775002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d80cfde0 0x7fe8d80d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8dd329640 1 -- 172.21.15.97:0/1797775002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d80cfde0 msgr2=0x7fe8d80d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:06.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8dd329640 1 --2- 172.21.15.97:0/1797775002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d80cfde0 0x7fe8d80d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8dd329640 1 -- 172.21.15.97:0/1797775002 >> v1:172.21.15.154:6789/0 conn(0x7fe8d80d0f90 legacy=0x7fe8d806c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8d7fff640 1 -- 172.21.15.97:0/1797775002 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 100572737 0 0) 0x7fe8c8001490 con 0x7fe8d80d0f90 2024-03-29T04:08:06.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.760+0000 7fe8dd329640 1 -- 172.21.15.97:0/1797775002 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d80d2480 con 0x7fe8d815e920 2024-03-29T04:08:06.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.761+0000 7fe8dd329640 1 --2- 172.21.15.97:0/1797775002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d815e920 0x7fe8d80d3240 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00098a0 tx=0x7fe8d002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca6da1c49db24f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:06.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.761+0000 7fe8d7fff640 1 -- 172.21.15.97:0/1797775002 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0034070 con 0x7fe8d815e920 2024-03-29T04:08:06.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.761+0000 7fe8d7fff640 1 -- 172.21.15.97:0/1797775002 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d0030840 con 0x7fe8d815e920 2024-03-29T04:08:06.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.761+0000 7fe8d7fff640 1 -- 172.21.15.97:0/1797775002 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0030c10 con 0x7fe8d815e920 2024-03-29T04:08:06.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.761+0000 7fe8df58d640 1 -- 172.21.15.97:0/1797775002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d815e920 msgr2=0x7fe8d80d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.761+0000 7fe8df58d640 1 --2- 172.21.15.97:0/1797775002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d815e920 0x7fe8d80d3240 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00098a0 tx=0x7fe8d002b980 comp rx=0 tx=0).stop 2024-03-29T04:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.761+0000 7fe8df58d640 1 -- 172.21.15.97:0/1797775002 shutdown_connections 2024-03-29T04:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.761+0000 7fe8df58d640 1 --2- 172.21.15.97:0/1797775002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d80cfde0 0x7fe8d80d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.761+0000 7fe8df58d640 1 --2- 172.21.15.97:0/1797775002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d815e920 0x7fe8d80d3240 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.761+0000 7fe8df58d640 1 -- 172.21.15.97:0/1797775002 >> 172.21.15.97:0/1797775002 conn(0x7fe8d808bfc0 msgr2=0x7fe8d80d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.762+0000 7fe8df58d640 1 -- 172.21.15.97:0/1797775002 shutdown_connections 2024-03-29T04:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.762+0000 7fe8df58d640 1 -- 172.21.15.97:0/1797775002 wait complete. 2024-03-29T04:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.763+0000 7fe8df58d640 1 Processor -- start 2024-03-29T04:08:06.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.763+0000 7fe8df58d640 1 -- start start 2024-03-29T04:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.763+0000 7fe8df58d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d80cfde0 0x7fe8d80cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.763+0000 7fe8df58d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d80d0f90 0x7fe8d81568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.763+0000 7fe8df58d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d815e920 0x7fe8d8159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.763+0000 7fe8df58d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d80d2d20 con 0x7fe8d815e920 2024-03-29T04:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.763+0000 7fe8df58d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d80d2a40 con 0x7fe8d80cfde0 2024-03-29T04:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.763+0000 7fe8df58d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d80d4ef0 con 0x7fe8d80d0f90 2024-03-29T04:08:06.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.763+0000 7fe8ddb2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d815e920 0x7fe8d8159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8ddb2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d815e920 0x7fe8d8159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39612/0 (socket says 172.21.15.97:39612) 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8ddb2a640 1 -- 172.21.15.97:0/138154685 learned_addr learned my addr 172.21.15.97:0/138154685 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8dcb28640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d80d0f90 0x7fe8d81568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8dd329640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d80cfde0 0x7fe8d80cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8ddb2a640 1 -- 172.21.15.97:0/138154685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d80d0f90 msgr2=0x7fe8d81568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8ddb2a640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d80d0f90 0x7fe8d81568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8ddb2a640 1 -- 172.21.15.97:0/138154685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d80cfde0 msgr2=0x7fe8d80cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8ddb2a640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d80cfde0 0x7fe8d80cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8ddb2a640 1 -- 172.21.15.97:0/138154685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d80cda30 con 0x7fe8d815e920 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8ddb2a640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d815e920 0x7fe8d8159f70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800c820 tx=0x7fe8c800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8d5ffb640 1 -- 172.21.15.97:0/138154685 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8018070 con 0x7fe8d815e920 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8d5ffb640 1 -- 172.21.15.97:0/138154685 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c800a5e0 con 0x7fe8d815e920 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8df58d640 1 -- 172.21.15.97:0/138154685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d80cdd10 con 0x7fe8d815e920 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8d5ffb640 1 -- 172.21.15.97:0/138154685 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c800a9d0 con 0x7fe8d815e920 2024-03-29T04:08:06.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.764+0000 7fe8df58d640 1 -- 172.21.15.97:0/138154685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d80ce2d0 con 0x7fe8d815e920 2024-03-29T04:08:06.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.765+0000 7fe8df58d640 1 -- 172.21.15.97:0/138154685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d80a2560 con 0x7fe8d815e920 2024-03-29T04:08:06.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.766+0000 7fe8d5ffb640 1 -- 172.21.15.97:0/138154685 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe8c801b070 con 0x7fe8d815e920 2024-03-29T04:08:06.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.766+0000 7fe8d5ffb640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe8c406c530 0x7fe8c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:06.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.766+0000 7fe8d5ffb640 1 -- 172.21.15.97:0/138154685 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe8c808abe0 con 0x7fe8d815e920 2024-03-29T04:08:06.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.766+0000 7fe8dd329640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe8c406c530 0x7fe8c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:06.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.766+0000 7fe8dd329640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe8c406c530 0x7fe8c406e9f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0036000 tx=0x7fe8d0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:06.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.770+0000 7fe8d5ffb640 1 -- 172.21.15.97:0/138154685 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe8d80d3780 con 0x7fe8d815e920 2024-03-29T04:08:06.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.946+0000 7fe8df58d640 1 -- 172.21.15.97:0/138154685 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe8d80c7ee0 con 0x7fe8d815e920 2024-03-29T04:08:06.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.947+0000 7fe8d5ffb640 1 -- 172.21.15.97:0/138154685 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fe8c8058c40 con 0x7fe8d815e920 2024-03-29T04:08:06.927 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:08:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.949+0000 7fe8df58d640 1 -- 172.21.15.97:0/138154685 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe8c406c530 msgr2=0x7fe8c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.949+0000 7fe8df58d640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe8c406c530 0x7fe8c406e9f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0036000 tx=0x7fe8d0008030 comp rx=0 tx=0).stop 2024-03-29T04:08:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.949+0000 7fe8df58d640 1 -- 172.21.15.97:0/138154685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d815e920 msgr2=0x7fe8d8159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.949+0000 7fe8df58d640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d815e920 0x7fe8d8159f70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800c820 tx=0x7fe8c800cce0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.949+0000 7fe8df58d640 1 -- 172.21.15.97:0/138154685 shutdown_connections 2024-03-29T04:08:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.949+0000 7fe8df58d640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe8c406c530 0x7fe8c406e9f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.949+0000 7fe8df58d640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d815e920 0x7fe8d8159f70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.949+0000 7fe8df58d640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe8d80d0f90 0x7fe8d81568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.949+0000 7fe8df58d640 1 --2- 172.21.15.97:0/138154685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe8d80cfde0 0x7fe8d80cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.949+0000 7fe8df58d640 1 -- 172.21.15.97:0/138154685 >> 172.21.15.97:0/138154685 conn(0x7fe8d808bfc0 msgr2=0x7fe8d81610c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.949+0000 7fe8df58d640 1 -- 172.21.15.97:0/138154685 shutdown_connections 2024-03-29T04:08:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:06.949+0000 7fe8df58d640 1 -- 172.21.15.97:0/138154685 wait complete. 2024-03-29T04:08:06.945 INFO:tasks.cephfs_test_runner:Starting test: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:08:06.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:07.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.066+0000 7f1e9c9ca640 1 Processor -- start 2024-03-29T04:08:07.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.066+0000 7f1e9c9ca640 1 -- start start 2024-03-29T04:08:07.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.067+0000 7f1e9c9ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e980b3c50 0x7f1e980d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.067+0000 7f1e9c9ca640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e980d3140 0x7f1e9806cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.067+0000 7f1e9c9ca640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e980cfb30 con 0x7f1e9815e930 2024-03-29T04:08:07.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.067+0000 7f1e9c9ca640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f1e9806c780 con 0x7f1e980d3140 2024-03-29T04:08:07.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.067+0000 7f1e9c9ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1e980cfe10 con 0x7f1e980b3c50 2024-03-29T04:08:07.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.067+0000 7f1e96d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e980d3140 0x7f1e9806cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.067+0000 7f1e96d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e980d3140 0x7f1e9806cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39624/0 (socket says 172.21.15.97:39624) 2024-03-29T04:08:07.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.067+0000 7f1e96d9d640 1 -- 172.21.15.97:0/1328934936 learned_addr learned my addr 172.21.15.97:0/1328934936 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:07.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.067+0000 7f1e95d9b640 1 --2- 172.21.15.97:0/1328934936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e980b3c50 0x7f1e980d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.068+0000 7f1e96d9d640 1 -- 172.21.15.97:0/1328934936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e980b3c50 msgr2=0x7f1e980d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:07.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.068+0000 7f1e96d9d640 1 --2- 172.21.15.97:0/1328934936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e980b3c50 0x7f1e980d2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.068+0000 7f1e96d9d640 1 -- 172.21.15.97:0/1328934936 >> v1:172.21.15.154:6789/0 conn(0x7f1e9815e930 legacy=0x7f1e980cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.068+0000 7f1e96d9d640 1 -- 172.21.15.97:0/1328934936 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e980d00f0 con 0x7f1e980d3140 2024-03-29T04:08:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.068+0000 7f1e9559a640 1 -- 172.21.15.97:0/1328934936 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1898550822 0 0) 0x7f1e980cfb30 con 0x7f1e9815e930 2024-03-29T04:08:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.068+0000 7f1e96d9d640 1 --2- 172.21.15.97:0/1328934936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e980d3140 0x7f1e9806cc10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009870 tx=0x7f1e9002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49807eb40033722 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.069+0000 7f1e9559a640 1 -- 172.21.15.97:0/1328934936 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90035020 con 0x7f1e980d3140 2024-03-29T04:08:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.069+0000 7f1e9559a640 1 -- 172.21.15.97:0/1328934936 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e90031640 con 0x7f1e980d3140 2024-03-29T04:08:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.069+0000 7f1e9559a640 1 -- 172.21.15.97:0/1328934936 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9002d040 con 0x7f1e980d3140 2024-03-29T04:08:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.069+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/1328934936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e980d3140 msgr2=0x7f1e9806cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.069+0000 7f1e9c9ca640 1 --2- 172.21.15.97:0/1328934936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e980d3140 0x7f1e9806cc10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009870 tx=0x7f1e9002b8f0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.069+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/1328934936 shutdown_connections 2024-03-29T04:08:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.069+0000 7f1e9c9ca640 1 --2- 172.21.15.97:0/1328934936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e980d3140 0x7f1e9806cc10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.069+0000 7f1e9c9ca640 1 --2- 172.21.15.97:0/1328934936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e980b3c50 0x7f1e980d2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.069+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/1328934936 >> 172.21.15.97:0/1328934936 conn(0x7f1e9808bfc0 msgr2=0x7f1e980d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.069+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/1328934936 shutdown_connections 2024-03-29T04:08:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.069+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/1328934936 wait complete. 2024-03-29T04:08:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.070+0000 7f1e9c9ca640 1 Processor -- start 2024-03-29T04:08:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.071+0000 7f1e9c9ca640 1 -- start start 2024-03-29T04:08:07.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.071+0000 7f1e9c9ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e980b3c50 0x7f1e9815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.071+0000 7f1e9c9ca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e980d3140 0x7f1e98155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.071+0000 7f1e9c9ca640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e9815e930 0x7f1e98157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.071+0000 7f1e9c9ca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9815ed10 con 0x7f1e980b3c50 2024-03-29T04:08:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.071+0000 7f1e9c9ca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1e980cf840 con 0x7f1e980d3140 2024-03-29T04:08:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.071+0000 7f1e9c9ca640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9815ee90 con 0x7f1e9815e930 2024-03-29T04:08:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.071+0000 7f1e9659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e980b3c50 0x7f1e9815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.071+0000 7f1e9659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e980b3c50 0x7f1e9815a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39640/0 (socket says 172.21.15.97:39640) 2024-03-29T04:08:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.071+0000 7f1e9659c640 1 -- 172.21.15.97:0/2913350377 learned_addr learned my addr 172.21.15.97:0/2913350377 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.071+0000 7f1e95d9b640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e980d3140 0x7f1e98155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.071+0000 7f1e96d9d640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e9815e930 0x7f1e98157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.072+0000 7f1e9659c640 1 -- 172.21.15.97:0/2913350377 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e9815e930 msgr2=0x7f1e98157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.072+0000 7f1e9659c640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e9815e930 0x7f1e98157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.072+0000 7f1e9659c640 1 -- 172.21.15.97:0/2913350377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e980d3140 msgr2=0x7f1e98155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.072+0000 7f1e9659c640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e980d3140 0x7f1e98155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.072+0000 7f1e9659c640 1 -- 172.21.15.97:0/2913350377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e98155fc0 con 0x7f1e980b3c50 2024-03-29T04:08:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.072+0000 7f1e9659c640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e980b3c50 0x7f1e9815a7f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800c7f0 tx=0x7f1e8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.072+0000 7f1e86ffd640 1 -- 172.21.15.97:0/2913350377 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88017070 con 0x7f1e980b3c50 2024-03-29T04:08:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.072+0000 7f1e86ffd640 1 -- 172.21.15.97:0/2913350377 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e88003da0 con 0x7f1e980b3c50 2024-03-29T04:08:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.072+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/2913350377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e981562a0 con 0x7f1e980b3c50 2024-03-29T04:08:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.072+0000 7f1e86ffd640 1 -- 172.21.15.97:0/2913350377 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88005290 con 0x7f1e980b3c50 2024-03-29T04:08:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.072+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/2913350377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e98156860 con 0x7f1e980b3c50 2024-03-29T04:08:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.073+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/2913350377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e98068640 con 0x7f1e980b3c50 2024-03-29T04:08:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.073+0000 7f1e86ffd640 1 -- 172.21.15.97:0/2913350377 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1e8801a070 con 0x7f1e980b3c50 2024-03-29T04:08:07.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.074+0000 7f1e86ffd640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1e7806c530 0x7f1e7806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.074+0000 7f1e86ffd640 1 -- 172.21.15.97:0/2913350377 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1e8808acc0 con 0x7f1e980b3c50 2024-03-29T04:08:07.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.074+0000 7f1e95d9b640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1e7806c530 0x7f1e7806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.074+0000 7f1e95d9b640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1e7806c530 0x7f1e7806e9f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e98161980 tx=0x7f1e8c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:07.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.078+0000 7f1e86ffd640 1 -- 172.21.15.97:0/2913350377 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1e88058c90 con 0x7f1e980b3c50 2024-03-29T04:08:07.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.252+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/2913350377 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e9806dd60 con 0x7f1e980b3c50 2024-03-29T04:08:07.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.253+0000 7f1e86ffd640 1 -- 172.21.15.97:0/2913350377 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f1e8805cf90 con 0x7f1e980b3c50 2024-03-29T04:08:07.234 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:07.234 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:07.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.254+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/2913350377 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1e7806c530 msgr2=0x7f1e7806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.254+0000 7f1e9c9ca640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1e7806c530 0x7f1e7806e9f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e98161980 tx=0x7f1e8c009190 comp rx=0 tx=0).stop 2024-03-29T04:08:07.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.255+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/2913350377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e980b3c50 msgr2=0x7f1e9815a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.255+0000 7f1e9c9ca640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e980b3c50 0x7f1e9815a7f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800c7f0 tx=0x7f1e8800ccb0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.255+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/2913350377 shutdown_connections 2024-03-29T04:08:07.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.255+0000 7f1e9c9ca640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1e7806c530 0x7f1e7806e9f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.255+0000 7f1e9c9ca640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e9815e930 0x7f1e98157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.255+0000 7f1e9c9ca640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e980d3140 0x7f1e98155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.255+0000 7f1e9c9ca640 1 --2- 172.21.15.97:0/2913350377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e980b3c50 0x7f1e9815a7f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.255+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/2913350377 >> 172.21.15.97:0/2913350377 conn(0x7f1e9808bfc0 msgr2=0x7f1e980d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:07.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.255+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/2913350377 shutdown_connections 2024-03-29T04:08:07.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.255+0000 7f1e9c9ca640 1 -- 172.21.15.97:0/2913350377 wait complete. 2024-03-29T04:08:07.246 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:07:38.618704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:07:39.724339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:08:07.246 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:08:07.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.368+0000 7fb36147e640 1 Processor -- start 2024-03-29T04:08:07.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.368+0000 7fb36147e640 1 -- start start 2024-03-29T04:08:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb36147e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb35c15e920 0x7fb35c0d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb36147e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35c0d3780 con 0x7fb35c0cfde0 2024-03-29T04:08:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb36147e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35c15ed00 con 0x7fb35c0d16a0 2024-03-29T04:08:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb36147e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fb35c0d21e0 con 0x7fb35c15e920 2024-03-29T04:08:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb35a7fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb35c0cfde0 0x7fb35c0d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33536/0 (socket says 172.21.15.97:33536) 2024-03-29T04:08:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb35a7fc640 1 -- 172.21.15.97:0/3899677427 learned_addr learned my addr 172.21.15.97:0/3899677427 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb35affd640 1 --2- 172.21.15.97:0/3899677427 >> v2:172.21.15.154:3300/0 conn(0x7fb35c15e920 0x7fb35c0d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb359ffb640 1 -- 172.21.15.97:0/3899677427 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2324715483 0 0) 0x7fb350001110 con 0x7fb35c0cfde0 2024-03-29T04:08:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb359ffb640 1 -- 172.21.15.97:0/3899677427 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb344003520 con 0x7fb35c0cfde0 2024-03-29T04:08:07.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb359ffb640 1 -- 172.21.15.97:0/3899677427 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 833483501 0 0) 0x7fb35c15ed00 con 0x7fb35c0d16a0 2024-03-29T04:08:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb359ffb640 1 -- 172.21.15.97:0/3899677427 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb350001110 con 0x7fb35c0d16a0 2024-03-29T04:08:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb35affd640 1 -- 172.21.15.97:0/3899677427 >> v1:172.21.15.154:6789/0 conn(0x7fb35c0d16a0 legacy=0x7fb35c06c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb35affd640 1 -- 172.21.15.97:0/3899677427 >> v1:172.21.15.97:6789/0 conn(0x7fb35c0cfde0 legacy=0x7fb35c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.369+0000 7fb35affd640 1 -- 172.21.15.97:0/3899677427 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c0d2480 con 0x7fb35c15e920 2024-03-29T04:08:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.370+0000 7fb35affd640 1 --2- 172.21.15.97:0/3899677427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb35c15e920 0x7fb35c0d3240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0097a0 tx=0x7fb34c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a11a108e6804257c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.370+0000 7fb359ffb640 1 -- 172.21.15.97:0/3899677427 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c034070 con 0x7fb35c15e920 2024-03-29T04:08:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.370+0000 7fb359ffb640 1 -- 172.21.15.97:0/3899677427 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34c030610 con 0x7fb35c15e920 2024-03-29T04:08:07.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.370+0000 7fb359ffb640 1 -- 172.21.15.97:0/3899677427 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0309e0 con 0x7fb35c15e920 2024-03-29T04:08:07.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.371+0000 7fb36147e640 1 -- 172.21.15.97:0/3899677427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb35c15e920 msgr2=0x7fb35c0d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.371+0000 7fb36147e640 1 --2- 172.21.15.97:0/3899677427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb35c15e920 0x7fb35c0d3240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0097a0 tx=0x7fb34c02b960 comp rx=0 tx=0).stop 2024-03-29T04:08:07.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.371+0000 7fb36147e640 1 -- 172.21.15.97:0/3899677427 shutdown_connections 2024-03-29T04:08:07.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.371+0000 7fb36147e640 1 --2- 172.21.15.97:0/3899677427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb35c15e920 0x7fb35c0d3240 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.371+0000 7fb36147e640 1 -- 172.21.15.97:0/3899677427 >> 172.21.15.97:0/3899677427 conn(0x7fb35c08bfc0 msgr2=0x7fb35c0d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:07.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.371+0000 7fb36147e640 1 -- 172.21.15.97:0/3899677427 shutdown_connections 2024-03-29T04:08:07.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.371+0000 7fb36147e640 1 -- 172.21.15.97:0/3899677427 wait complete. 2024-03-29T04:08:07.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.372+0000 7fb36147e640 1 Processor -- start 2024-03-29T04:08:07.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.372+0000 7fb36147e640 1 -- start start 2024-03-29T04:08:07.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb36147e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb35c0cfde0 0x7fb35c0ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb36147e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb35c0d16a0 0x7fb35c0b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb36147e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb35c15e920 0x7fb35c0cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb36147e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c0d2d20 con 0x7fb35c0d16a0 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb36147e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c0d2a40 con 0x7fb35c0cfde0 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb36147e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb35c0d4ef0 con 0x7fb35c15e920 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb35a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb35c0d16a0 0x7fb35c0b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb35a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb35c0d16a0 0x7fb35c0b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39650/0 (socket says 172.21.15.97:39650) 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb35a7fc640 1 -- 172.21.15.97:0/3474133123 learned_addr learned my addr 172.21.15.97:0/3474133123 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb35affd640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb35c0cfde0 0x7fb35c0ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb35b7fe640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb35c15e920 0x7fb35c0cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb35a7fc640 1 -- 172.21.15.97:0/3474133123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb35c15e920 msgr2=0x7fb35c0cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb35a7fc640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb35c15e920 0x7fb35c0cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb35a7fc640 1 -- 172.21.15.97:0/3474133123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb35c0cfde0 msgr2=0x7fb35c0ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb35a7fc640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb35c0cfde0 0x7fb35c0ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb35a7fc640 1 -- 172.21.15.97:0/3474133123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c0cbac0 con 0x7fb35c0d16a0 2024-03-29T04:08:07.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb35affd640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb35c0cfde0 0x7fb35c0ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb35a7fc640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb35c0d16a0 0x7fb35c0b1ea0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb35000c730 tx=0x7fb35000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.373+0000 7fb3437fe640 1 -- 172.21.15.97:0/3474133123 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350017070 con 0x7fb35c0d16a0 2024-03-29T04:08:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.374+0000 7fb3437fe640 1 -- 172.21.15.97:0/3474133123 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb350005af0 con 0x7fb35c0d16a0 2024-03-29T04:08:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.374+0000 7fb3437fe640 1 -- 172.21.15.97:0/3474133123 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350007070 con 0x7fb35c0d16a0 2024-03-29T04:08:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.374+0000 7fb36147e640 1 -- 172.21.15.97:0/3474133123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35c0cbda0 con 0x7fb35c0d16a0 2024-03-29T04:08:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.374+0000 7fb36147e640 1 -- 172.21.15.97:0/3474133123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35c0cc360 con 0x7fb35c0d16a0 2024-03-29T04:08:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.375+0000 7fb36147e640 1 -- 172.21.15.97:0/3474133123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35c0a2560 con 0x7fb35c0d16a0 2024-03-29T04:08:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.375+0000 7fb3437fe640 1 -- 172.21.15.97:0/3474133123 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb35001a070 con 0x7fb35c0d16a0 2024-03-29T04:08:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.375+0000 7fb3437fe640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb34806c530 0x7fb34806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.375+0000 7fb3437fe640 1 -- 172.21.15.97:0/3474133123 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb35008aa70 con 0x7fb35c0d16a0 2024-03-29T04:08:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.375+0000 7fb35affd640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb34806c530 0x7fb34806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.376+0000 7fb35affd640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb34806c530 0x7fb34806e9f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb34c036000 tx=0x7fb34c03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:07.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.380+0000 7fb3437fe640 1 -- 172.21.15.97:0/3474133123 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb35c0d3780 con 0x7fb35c0d16a0 2024-03-29T04:08:07.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.554+0000 7fb36147e640 1 -- 172.21.15.97:0/3474133123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb35c04f9d0 con 0x7fb35c0d16a0 2024-03-29T04:08:07.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.554+0000 7fb3437fe640 1 -- 172.21.15.97:0/3474133123 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb350058b90 con 0x7fb35c0d16a0 2024-03-29T04:08:07.534 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:08:07.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.556+0000 7fb36147e640 1 -- 172.21.15.97:0/3474133123 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb34806c530 msgr2=0x7fb34806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.556+0000 7fb36147e640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb34806c530 0x7fb34806e9f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb34c036000 tx=0x7fb34c03b040 comp rx=0 tx=0).stop 2024-03-29T04:08:07.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.556+0000 7fb36147e640 1 -- 172.21.15.97:0/3474133123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb35c0d16a0 msgr2=0x7fb35c0b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.556+0000 7fb36147e640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb35c0d16a0 0x7fb35c0b1ea0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb35000c730 tx=0x7fb35000cbf0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.556+0000 7fb36147e640 1 -- 172.21.15.97:0/3474133123 shutdown_connections 2024-03-29T04:08:07.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.556+0000 7fb36147e640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb34806c530 0x7fb34806e9f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.556+0000 7fb36147e640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb35c15e920 0x7fb35c0cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.556+0000 7fb36147e640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb35c0d16a0 0x7fb35c0b1ea0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.556+0000 7fb36147e640 1 --2- 172.21.15.97:0/3474133123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb35c0cfde0 0x7fb35c0ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.556+0000 7fb36147e640 1 -- 172.21.15.97:0/3474133123 >> 172.21.15.97:0/3474133123 conn(0x7fb35c08bfc0 msgr2=0x7fb35c0b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:07.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.557+0000 7fb36147e640 1 -- 172.21.15.97:0/3474133123 shutdown_connections 2024-03-29T04:08:07.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.557+0000 7fb36147e640 1 -- 172.21.15.97:0/3474133123 wait complete. 2024-03-29T04:08:07.547 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:08:07.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.671+0000 7f71519af640 1 Processor -- start 2024-03-29T04:08:07.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.671+0000 7f71519af640 1 -- start start 2024-03-29T04:08:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f71519af640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f714c15e8b0 0x7f714c0b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f71519af640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f714c06b510 0x7f714c06b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f71519af640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f714c0b6500 con 0x7f714c0b6020 2024-03-29T04:08:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f71519af640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f714c0b67e0 con 0x7f714c06b510 2024-03-29T04:08:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f71519af640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f714c0b4470 con 0x7f714c15e8b0 2024-03-29T04:08:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f714b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f714c06b510 0x7f714c06b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f714b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f714c06b510 0x7f714c06b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39656/0 (socket says 172.21.15.97:39656) 2024-03-29T04:08:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f714a7fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f714c0b6020 0x7f714c0b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33552/0 (socket says 172.21.15.97:33552) 2024-03-29T04:08:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f714b7fe640 1 -- 172.21.15.97:0/2920340658 learned_addr learned my addr 172.21.15.97:0/2920340658 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f714affd640 1 --2- 172.21.15.97:0/2920340658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f714c15e8b0 0x7f714c0b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f714b7fe640 1 -- 172.21.15.97:0/2920340658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f714c15e8b0 msgr2=0x7f714c0b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f714b7fe640 1 --2- 172.21.15.97:0/2920340658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f714c15e8b0 0x7f714c0b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f714b7fe640 1 -- 172.21.15.97:0/2920340658 >> v1:172.21.15.97:6789/0 conn(0x7f714c0b6020 legacy=0x7f714c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f7149ffb640 1 -- 172.21.15.97:0/2920340658 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 659077833 0 0) 0x7f714c0b6500 con 0x7f714c0b6020 2024-03-29T04:08:07.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.672+0000 7f714b7fe640 1 -- 172.21.15.97:0/2920340658 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c0b4750 con 0x7f714c06b510 2024-03-29T04:08:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.673+0000 7f714b7fe640 1 --2- 172.21.15.97:0/2920340658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714c06b510 0x7f714c06b9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7144009e50 tx=0x7f714402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9519dacd549e64a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.673+0000 7f7149ffb640 1 -- 172.21.15.97:0/2920340658 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714402ca70 con 0x7f714c06b510 2024-03-29T04:08:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.673+0000 7f7149ffb640 1 -- 172.21.15.97:0/2920340658 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f714402cc10 con 0x7f714c06b510 2024-03-29T04:08:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.673+0000 7f7149ffb640 1 -- 172.21.15.97:0/2920340658 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71440343f0 con 0x7f714c06b510 2024-03-29T04:08:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.674+0000 7f71519af640 1 -- 172.21.15.97:0/2920340658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714c06b510 msgr2=0x7f714c06b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.674+0000 7f71519af640 1 --2- 172.21.15.97:0/2920340658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714c06b510 0x7f714c06b9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7144009e50 tx=0x7f714402bed0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.674+0000 7f71519af640 1 -- 172.21.15.97:0/2920340658 shutdown_connections 2024-03-29T04:08:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.674+0000 7f71519af640 1 --2- 172.21.15.97:0/2920340658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714c06b510 0x7f714c06b9c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.674+0000 7f71519af640 1 --2- 172.21.15.97:0/2920340658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f714c15e8b0 0x7f714c0b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.674+0000 7f71519af640 1 -- 172.21.15.97:0/2920340658 >> 172.21.15.97:0/2920340658 conn(0x7f714c072e40 msgr2=0x7f714c0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.674+0000 7f71519af640 1 -- 172.21.15.97:0/2920340658 shutdown_connections 2024-03-29T04:08:07.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.674+0000 7f71519af640 1 -- 172.21.15.97:0/2920340658 wait complete. 2024-03-29T04:08:07.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.675+0000 7f71519af640 1 Processor -- start 2024-03-29T04:08:07.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.675+0000 7f71519af640 1 -- start start 2024-03-29T04:08:07.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f71519af640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f714c06b510 0x7f714c14ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f71519af640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f714c0b6020 0x7f714c1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f71519af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714c15e8b0 0x7f714c160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f71519af640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f714c06c190 con 0x7f714c15e8b0 2024-03-29T04:08:07.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f71519af640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f714c0b4b30 con 0x7f714c0b6020 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f71519af640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f714c06c310 con 0x7f714c06b510 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f714b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714c15e8b0 0x7f714c160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f714b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714c15e8b0 0x7f714c160fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39668/0 (socket says 172.21.15.97:39668) 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f714b7fe640 1 -- 172.21.15.97:0/2783428072 learned_addr learned my addr 172.21.15.97:0/2783428072 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f714a7fc640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f714c0b6020 0x7f714c1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f714affd640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f714c06b510 0x7f714c14ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f714b7fe640 1 -- 172.21.15.97:0/2783428072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f714c06b510 msgr2=0x7f714c14ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f714b7fe640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f714c06b510 0x7f714c14ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f714b7fe640 1 -- 172.21.15.97:0/2783428072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f714c0b6020 msgr2=0x7f714c1564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f714b7fe640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f714c0b6020 0x7f714c1564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f714b7fe640 1 -- 172.21.15.97:0/2783428072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c14b180 con 0x7f714c15e8b0 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f714b7fe640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714c15e8b0 0x7f714c160fa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7144036040 tx=0x7f7144004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f7133fff640 1 -- 172.21.15.97:0/2783428072 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144043030 con 0x7f714c15e8b0 2024-03-29T04:08:07.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f7133fff640 1 -- 172.21.15.97:0/2783428072 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f714403e450 con 0x7f714c15e8b0 2024-03-29T04:08:07.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f7133fff640 1 -- 172.21.15.97:0/2783428072 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144009510 con 0x7f714c15e8b0 2024-03-29T04:08:07.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f71519af640 1 -- 172.21.15.97:0/2783428072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714c158a90 con 0x7f714c15e8b0 2024-03-29T04:08:07.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.676+0000 7f71519af640 1 -- 172.21.15.97:0/2783428072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714c159330 con 0x7f714c15e8b0 2024-03-29T04:08:07.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.677+0000 7f71519af640 1 -- 172.21.15.97:0/2783428072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f714c09f6a0 con 0x7f714c15e8b0 2024-03-29T04:08:07.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.678+0000 7f7133fff640 1 -- 172.21.15.97:0/2783428072 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7144046070 con 0x7f714c15e8b0 2024-03-29T04:08:07.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.678+0000 7f7133fff640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f713806c530 0x7f713806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.678+0000 7f7133fff640 1 -- 172.21.15.97:0/2783428072 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f71440ad8d0 con 0x7f714c15e8b0 2024-03-29T04:08:07.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.678+0000 7f714affd640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f713806c530 0x7f713806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.679+0000 7f714affd640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f713806c530 0x7f713806e9f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f713c009aa0 tx=0x7f713c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:07.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.683+0000 7f7133fff640 1 -- 172.21.15.97:0/2783428072 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f714404d070 con 0x7f714c15e8b0 2024-03-29T04:08:07.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.856+0000 7f71519af640 1 -- 172.21.15.97:0/2783428072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f714c0b0bc0 con 0x7f714c15e8b0 2024-03-29T04:08:07.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.857+0000 7f7133fff640 1 -- 172.21.15.97:0/2783428072 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f714407b8a0 con 0x7f714c15e8b0 2024-03-29T04:08:07.837 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:08:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.859+0000 7f71519af640 1 -- 172.21.15.97:0/2783428072 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f713806c530 msgr2=0x7f713806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.859+0000 7f71519af640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f713806c530 0x7f713806e9f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f713c009aa0 tx=0x7f713c009190 comp rx=0 tx=0).stop 2024-03-29T04:08:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.859+0000 7f71519af640 1 -- 172.21.15.97:0/2783428072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714c15e8b0 msgr2=0x7f714c160fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.859+0000 7f71519af640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714c15e8b0 0x7f714c160fa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7144036040 tx=0x7f7144004a40 comp rx=0 tx=0).stop 2024-03-29T04:08:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.859+0000 7f71519af640 1 -- 172.21.15.97:0/2783428072 shutdown_connections 2024-03-29T04:08:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.859+0000 7f71519af640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f713806c530 0x7f713806e9f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.859+0000 7f71519af640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f714c15e8b0 0x7f714c160fa0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.859+0000 7f71519af640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f714c0b6020 0x7f714c1564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.859+0000 7f71519af640 1 --2- 172.21.15.97:0/2783428072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f714c06b510 0x7f714c14ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.859+0000 7f71519af640 1 -- 172.21.15.97:0/2783428072 >> 172.21.15.97:0/2783428072 conn(0x7f714c072e40 msgr2=0x7f714c06b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.859+0000 7f71519af640 1 -- 172.21.15.97:0/2783428072 shutdown_connections 2024-03-29T04:08:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.859+0000 7f71519af640 1 -- 172.21.15.97:0/2783428072 wait complete. 2024-03-29T04:08:07.850 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:08:07.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.974+0000 7fd9993d5640 1 Processor -- start 2024-03-29T04:08:07.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.975+0000 7fd9993d5640 1 -- start start 2024-03-29T04:08:07.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.975+0000 7fd9993d5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd99415f6e0 0x7fd9940b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.975+0000 7fd9993d5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd994160140 con 0x7fd9940b2a70 2024-03-29T04:08:07.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.975+0000 7fd9993d5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9940b1b60 con 0x7fd9940cf3f0 2024-03-29T04:08:07.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.975+0000 7fd9993d5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd9940b03c0 con 0x7fd99415f6e0 2024-03-29T04:08:07.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.975+0000 7fd9927fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd9940b2a70 0x7fd9940afcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33566/0 (socket says 172.21.15.97:33566) 2024-03-29T04:08:07.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.975+0000 7fd9927fc640 1 -- 172.21.15.97:0/2007905492 learned_addr learned my addr 172.21.15.97:0/2007905492 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:07.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.975+0000 7fd992ffd640 1 --2- 172.21.15.97:0/2007905492 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd99415f6e0 0x7fd9940b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.976+0000 7fd991ffb640 1 -- 172.21.15.97:0/2007905492 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2688992643 0 0) 0x7fd994160140 con 0x7fd9940b2a70 2024-03-29T04:08:07.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.976+0000 7fd991ffb640 1 -- 172.21.15.97:0/2007905492 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd97c003520 con 0x7fd9940b2a70 2024-03-29T04:08:07.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.976+0000 7fd991ffb640 1 -- 172.21.15.97:0/2007905492 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1556958210 0 0) 0x7fd9940b1b60 con 0x7fd9940cf3f0 2024-03-29T04:08:07.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.976+0000 7fd991ffb640 1 -- 172.21.15.97:0/2007905492 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd994160140 con 0x7fd9940cf3f0 2024-03-29T04:08:07.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.976+0000 7fd992ffd640 1 -- 172.21.15.97:0/2007905492 >> v1:172.21.15.154:6789/0 conn(0x7fd9940cf3f0 legacy=0x7fd9940ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.976+0000 7fd992ffd640 1 -- 172.21.15.97:0/2007905492 >> v1:172.21.15.97:6789/0 conn(0x7fd9940b2a70 legacy=0x7fd9940afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.976+0000 7fd992ffd640 1 -- 172.21.15.97:0/2007905492 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9940b0540 con 0x7fd99415f6e0 2024-03-29T04:08:07.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.977+0000 7fd992ffd640 1 --2- 172.21.15.97:0/2007905492 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd99415f6e0 0x7fd9940b1e40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd9840097d0 tx=0x7fd98402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ef74efae0e89d27 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:07.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.977+0000 7fd991ffb640 1 -- 172.21.15.97:0/2007905492 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984035020 con 0x7fd99415f6e0 2024-03-29T04:08:07.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.977+0000 7fd991ffb640 1 -- 172.21.15.97:0/2007905492 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd984031800 con 0x7fd99415f6e0 2024-03-29T04:08:07.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.977+0000 7fd991ffb640 1 -- 172.21.15.97:0/2007905492 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984031bb0 con 0x7fd99415f6e0 2024-03-29T04:08:07.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.977+0000 7fd9993d5640 1 -- 172.21.15.97:0/2007905492 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd99415f6e0 msgr2=0x7fd9940b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.977+0000 7fd9993d5640 1 --2- 172.21.15.97:0/2007905492 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd99415f6e0 0x7fd9940b1e40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd9840097d0 tx=0x7fd98402b8a0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.977+0000 7fd9993d5640 1 -- 172.21.15.97:0/2007905492 shutdown_connections 2024-03-29T04:08:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.977+0000 7fd9993d5640 1 --2- 172.21.15.97:0/2007905492 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd99415f6e0 0x7fd9940b1e40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.977+0000 7fd9993d5640 1 -- 172.21.15.97:0/2007905492 >> 172.21.15.97:0/2007905492 conn(0x7fd9940b2ed0 msgr2=0x7fd9940b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.977+0000 7fd9993d5640 1 -- 172.21.15.97:0/2007905492 shutdown_connections 2024-03-29T04:08:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.977+0000 7fd9993d5640 1 -- 172.21.15.97:0/2007905492 wait complete. 2024-03-29T04:08:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.979+0000 7fd9993d5640 1 Processor -- start 2024-03-29T04:08:07.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.979+0000 7fd9993d5640 1 -- start start 2024-03-29T04:08:07.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.979+0000 7fd9993d5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9940b2a70 0x7fd994153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.979+0000 7fd9993d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9940cf3f0 0x7fd994153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.979+0000 7fd9993d5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd99415f6e0 0x7fd99415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.979+0000 7fd9993d5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd99415fe90 con 0x7fd9940cf3f0 2024-03-29T04:08:07.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.979+0000 7fd9993d5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd99415fd10 con 0x7fd99415f6e0 2024-03-29T04:08:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.979+0000 7fd9993d5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd9940aec30 con 0x7fd9940b2a70 2024-03-29T04:08:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.979+0000 7fd9927fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9940cf3f0 0x7fd994153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.979+0000 7fd9927fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9940cf3f0 0x7fd994153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39674/0 (socket says 172.21.15.97:39674) 2024-03-29T04:08:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.979+0000 7fd9927fc640 1 -- 172.21.15.97:0/1425464192 learned_addr learned my addr 172.21.15.97:0/1425464192 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.979+0000 7fd992ffd640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9940b2a70 0x7fd994153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.979+0000 7fd9937fe640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd99415f6e0 0x7fd99415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.980+0000 7fd9927fc640 1 -- 172.21.15.97:0/1425464192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9940b2a70 msgr2=0x7fd994153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.980+0000 7fd9927fc640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9940b2a70 0x7fd994153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.980+0000 7fd9927fc640 1 -- 172.21.15.97:0/1425464192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd99415f6e0 msgr2=0x7fd99415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.980+0000 7fd9927fc640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd99415f6e0 0x7fd99415c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.980+0000 7fd9927fc640 1 -- 172.21.15.97:0/1425464192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9940cac90 con 0x7fd9940cf3f0 2024-03-29T04:08:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.980+0000 7fd9927fc640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9940cf3f0 0x7fd994153960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd98800c500 tx=0x7fd98800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.980+0000 7fd97b7fe640 1 -- 172.21.15.97:0/1425464192 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988017070 con 0x7fd9940cf3f0 2024-03-29T04:08:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.980+0000 7fd97b7fe640 1 -- 172.21.15.97:0/1425464192 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd988005ce0 con 0x7fd9940cf3f0 2024-03-29T04:08:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.980+0000 7fd9993d5640 1 -- 172.21.15.97:0/1425464192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9940caf70 con 0x7fd9940cf3f0 2024-03-29T04:08:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.980+0000 7fd97b7fe640 1 -- 172.21.15.97:0/1425464192 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9880072f0 con 0x7fd9940cf3f0 2024-03-29T04:08:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.980+0000 7fd9993d5640 1 -- 172.21.15.97:0/1425464192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9941544e0 con 0x7fd9940cf3f0 2024-03-29T04:08:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.981+0000 7fd9993d5640 1 -- 172.21.15.97:0/1425464192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd99409f530 con 0x7fd9940cf3f0 2024-03-29T04:08:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.981+0000 7fd97b7fe640 1 -- 172.21.15.97:0/1425464192 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd98801a070 con 0x7fd9940cf3f0 2024-03-29T04:08:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.982+0000 7fd97b7fe640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd98006c530 0x7fd98006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.982+0000 7fd97b7fe640 1 -- 172.21.15.97:0/1425464192 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd98808aa00 con 0x7fd9940cf3f0 2024-03-29T04:08:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.982+0000 7fd992ffd640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd98006c530 0x7fd98006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.982+0000 7fd992ffd640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd98006c530 0x7fd98006e9f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd984036000 tx=0x7fd98403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:07.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:07.986+0000 7fd97b7fe640 1 -- 172.21.15.97:0/1425464192 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd9880589d0 con 0x7fd9940cf3f0 2024-03-29T04:08:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.158+0000 7fd9993d5640 1 -- 172.21.15.97:0/1425464192 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd9940cb710 con 0x7fd9940cf3f0 2024-03-29T04:08:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.158+0000 7fd97b7fe640 1 -- 172.21.15.97:0/1425464192 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fd98805ccd0 con 0x7fd9940cf3f0 2024-03-29T04:08:08.138 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:08:08.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.160+0000 7fd9993d5640 1 -- 172.21.15.97:0/1425464192 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd98006c530 msgr2=0x7fd98006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:08.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.160+0000 7fd9993d5640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd98006c530 0x7fd98006e9f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd984036000 tx=0x7fd98403c040 comp rx=0 tx=0).stop 2024-03-29T04:08:08.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.160+0000 7fd9993d5640 1 -- 172.21.15.97:0/1425464192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9940cf3f0 msgr2=0x7fd994153960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:08.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.160+0000 7fd9993d5640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9940cf3f0 0x7fd994153960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd98800c500 tx=0x7fd98800c9c0 comp rx=0 tx=0).stop 2024-03-29T04:08:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.160+0000 7fd9993d5640 1 -- 172.21.15.97:0/1425464192 shutdown_connections 2024-03-29T04:08:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.160+0000 7fd9993d5640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd98006c530 0x7fd98006e9f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.160+0000 7fd9993d5640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd99415f6e0 0x7fd99415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.160+0000 7fd9993d5640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9940cf3f0 0x7fd994153960 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.160+0000 7fd9993d5640 1 --2- 172.21.15.97:0/1425464192 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd9940b2a70 0x7fd994153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.160+0000 7fd9993d5640 1 -- 172.21.15.97:0/1425464192 >> 172.21.15.97:0/1425464192 conn(0x7fd9940b2ed0 msgr2=0x7fd99415ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.160+0000 7fd9993d5640 1 -- 172.21.15.97:0/1425464192 shutdown_connections 2024-03-29T04:08:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.160+0000 7fd9993d5640 1 -- 172.21.15.97:0/1425464192 wait complete. 2024-03-29T04:08:08.151 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining' 2024-03-29T04:08:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.272+0000 7fa1f75af640 1 Processor -- start 2024-03-29T04:08:08.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.272+0000 7fa1f75af640 1 -- start start 2024-03-29T04:08:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1f75af640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1f006e440 0x7fa1f00ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:08.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1f75af640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa1f006fb60 0x7fa1f006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1f75af640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f015ec50 con 0x7fa1f00b3af0 2024-03-29T04:08:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1f75af640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f00b4070 con 0x7fa1f006fb60 2024-03-29T04:08:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1f75af640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f006b5b0 con 0x7fa1f006e440 2024-03-29T04:08:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1f5b4c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa1f006fb60 0x7fa1f006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1f4b4a640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa1f00b3af0 0x7fa1f006e820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33572/0 (socket says 172.21.15.97:33572) 2024-03-29T04:08:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1f4b4a640 1 -- 172.21.15.97:0/2216900486 learned_addr learned my addr 172.21.15.97:0/2216900486 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1f534b640 1 --2- 172.21.15.97:0/2216900486 >> v2:172.21.15.154:3300/0 conn(0x7fa1f006e440 0x7fa1f00ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1f5b4c640 1 -- 172.21.15.97:0/2216900486 >> v2:172.21.15.154:3300/0 conn(0x7fa1f006e440 msgr2=0x7fa1f00ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1f5b4c640 1 --2- 172.21.15.97:0/2216900486 >> v2:172.21.15.154:3300/0 conn(0x7fa1f006e440 0x7fa1f00ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1f5b4c640 1 -- 172.21.15.97:0/2216900486 >> v1:172.21.15.97:6789/0 conn(0x7fa1f00b3af0 legacy=0x7fa1f006e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:08.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1e7fff640 1 -- 172.21.15.97:0/2216900486 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1243200190 0 0) 0x7fa1f015ec50 con 0x7fa1f00b3af0 2024-03-29T04:08:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.273+0000 7fa1f5b4c640 1 -- 172.21.15.97:0/2216900486 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f006b890 con 0x7fa1f006fb60 2024-03-29T04:08:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.274+0000 7fa1f5b4c640 1 --2- 172.21.15.97:0/2216900486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1f006fb60 0x7fa1f006ef30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00700e0 tx=0x7fa1e802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5768c0f9df438e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.274+0000 7fa1e7fff640 1 -- 172.21.15.97:0/2216900486 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8039070 con 0x7fa1f006fb60 2024-03-29T04:08:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.274+0000 7fa1e7fff640 1 -- 172.21.15.97:0/2216900486 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e802cbf0 con 0x7fa1f006fb60 2024-03-29T04:08:08.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.274+0000 7fa1e7fff640 1 -- 172.21.15.97:0/2216900486 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8034440 con 0x7fa1f006fb60 2024-03-29T04:08:08.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.274+0000 7fa1f75af640 1 -- 172.21.15.97:0/2216900486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1f006fb60 msgr2=0x7fa1f006ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:08.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.274+0000 7fa1f75af640 1 --2- 172.21.15.97:0/2216900486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1f006fb60 0x7fa1f006ef30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00700e0 tx=0x7fa1e802be40 comp rx=0 tx=0).stop 2024-03-29T04:08:08.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.275+0000 7fa1f75af640 1 -- 172.21.15.97:0/2216900486 shutdown_connections 2024-03-29T04:08:08.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.275+0000 7fa1f75af640 1 --2- 172.21.15.97:0/2216900486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1f006fb60 0x7fa1f006ef30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:08.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.275+0000 7fa1f75af640 1 --2- 172.21.15.97:0/2216900486 >> v2:172.21.15.154:3300/0 conn(0x7fa1f006e440 0x7fa1f00ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:08.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.275+0000 7fa1f75af640 1 -- 172.21.15.97:0/2216900486 >> 172.21.15.97:0/2216900486 conn(0x7fa1f00d0b20 msgr2=0x7fa1f00d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:08.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.275+0000 7fa1f75af640 1 -- 172.21.15.97:0/2216900486 shutdown_connections 2024-03-29T04:08:08.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.275+0000 7fa1f75af640 1 -- 172.21.15.97:0/2216900486 wait complete. 2024-03-29T04:08:08.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.276+0000 7fa1f75af640 1 Processor -- start 2024-03-29T04:08:08.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.276+0000 7fa1f75af640 1 -- start start 2024-03-29T04:08:08.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.276+0000 7fa1f75af640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f006e440 0x7fa1f01f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:08.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.276+0000 7fa1f75af640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1f006fb60 0x7fa1f0154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:08.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f75af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1f00b3af0 0x7fa1f014fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:08.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f75af640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00d1760 con 0x7fa1f00b3af0 2024-03-29T04:08:08.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f75af640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f006be50 con 0x7fa1f006e440 2024-03-29T04:08:08.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f75af640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f00adf20 con 0x7fa1f006fb60 2024-03-29T04:08:08.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f5b4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1f00b3af0 0x7fa1f014fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:08.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f5b4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1f00b3af0 0x7fa1f014fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39690/0 (socket says 172.21.15.97:39690) 2024-03-29T04:08:08.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f5b4c640 1 -- 172.21.15.97:0/1046136150 learned_addr learned my addr 172.21.15.97:0/1046136150 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:08.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f534b640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f006e440 0x7fa1f01f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:08.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f4b4a640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1f006fb60 0x7fa1f0154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:08.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f5b4c640 1 -- 172.21.15.97:0/1046136150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1f006fb60 msgr2=0x7fa1f0154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:08.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f5b4c640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1f006fb60 0x7fa1f0154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:08.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f5b4c640 1 -- 172.21.15.97:0/1046136150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f006e440 msgr2=0x7fa1f01f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:08.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f5b4c640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f006e440 0x7fa1f01f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:08.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f5b4c640 1 -- 172.21.15.97:0/1046136150 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f01f47b0 con 0x7fa1f00b3af0 2024-03-29T04:08:08.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f5b4c640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1f00b3af0 0x7fa1f014fd50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8037040 tx=0x7fa1e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:08.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1e5ffb640 1 -- 172.21.15.97:0/1046136150 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8044020 con 0x7fa1f00b3af0 2024-03-29T04:08:08.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1e5ffb640 1 -- 172.21.15.97:0/1046136150 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e803a040 con 0x7fa1f00b3af0 2024-03-29T04:08:08.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1e5ffb640 1 -- 172.21.15.97:0/1046136150 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8039050 con 0x7fa1f00b3af0 2024-03-29T04:08:08.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.277+0000 7fa1f75af640 1 -- 172.21.15.97:0/1046136150 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f01f4a90 con 0x7fa1f00b3af0 2024-03-29T04:08:08.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.278+0000 7fa1f75af640 1 -- 172.21.15.97:0/1046136150 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f0158ea0 con 0x7fa1f00b3af0 2024-03-29T04:08:08.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.279+0000 7fa1f75af640 1 -- 172.21.15.97:0/1046136150 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f0158cc0 con 0x7fa1f00b3af0 2024-03-29T04:08:08.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.279+0000 7fa1e5ffb640 1 -- 172.21.15.97:0/1046136150 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa1e8003b90 con 0x7fa1f00b3af0 2024-03-29T04:08:08.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.279+0000 7fa1e5ffb640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa1d406c530 0x7fa1d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:08.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.279+0000 7fa1e5ffb640 1 -- 172.21.15.97:0/1046136150 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa1e80adce0 con 0x7fa1f00b3af0 2024-03-29T04:08:08.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.279+0000 7fa1f534b640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa1d406c530 0x7fa1d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:08.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.280+0000 7fa1f534b640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa1d406c530 0x7fa1d406e9f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009aa0 tx=0x7fa1ec009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:08.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.284+0000 7fa1e5ffb640 1 -- 172.21.15.97:0/1046136150 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa1f0158cc0 con 0x7fa1f00b3af0 2024-03-29T04:08:08.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:08.525+0000 7fa1f75af640 1 -- 172.21.15.97:0/1046136150 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]} v 0) -- 0x7fa1f01590a0 con 0x7fa1f00b3af0 2024-03-29T04:08:09.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.517+0000 7fa1e5ffb640 1 -- 172.21.15.97:0/1046136150 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]}]=0 v52) ==== 150+0+0 (secure 0 0 0) 0x7fa1e804e020 con 0x7fa1f00b3af0 2024-03-29T04:08:09.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.520+0000 7fa1f75af640 1 -- 172.21.15.97:0/1046136150 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa1d406c530 msgr2=0x7fa1d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:09.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.520+0000 7fa1f75af640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa1d406c530 0x7fa1d406e9f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009aa0 tx=0x7fa1ec009190 comp rx=0 tx=0).stop 2024-03-29T04:08:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.520+0000 7fa1f75af640 1 -- 172.21.15.97:0/1046136150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1f00b3af0 msgr2=0x7fa1f014fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.520+0000 7fa1f75af640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1f00b3af0 0x7fa1f014fd50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8037040 tx=0x7fa1e8004a40 comp rx=0 tx=0).stop 2024-03-29T04:08:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.520+0000 7fa1f75af640 1 -- 172.21.15.97:0/1046136150 shutdown_connections 2024-03-29T04:08:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.520+0000 7fa1f75af640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa1d406c530 0x7fa1d406e9f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.520+0000 7fa1f75af640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1f00b3af0 0x7fa1f014fd50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.520+0000 7fa1f75af640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa1f006fb60 0x7fa1f0154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.520+0000 7fa1f75af640 1 --2- 172.21.15.97:0/1046136150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1f006e440 0x7fa1f01f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.520+0000 7fa1f75af640 1 -- 172.21.15.97:0/1046136150 >> 172.21.15.97:0/1046136150 conn(0x7fa1f00d0b20 msgr2=0x7fa1f006c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.520+0000 7fa1f75af640 1 -- 172.21.15.97:0/1046136150 shutdown_connections 2024-03-29T04:08:09.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.520+0000 7fa1f75af640 1 -- 172.21.15.97:0/1046136150 wait complete. 2024-03-29T04:08:09.512 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-29T04:08:09.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.637+0000 7f1f25d60640 1 Processor -- start 2024-03-29T04:08:09.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.637+0000 7f1f25d60640 1 -- start start 2024-03-29T04:08:09.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f25d60640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1f200ca950 0x7f1f200b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:09.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f25d60640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f200b26f0 0x7f1f2015be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:09.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f25d60640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f200cade0 con 0x7f1f2015b3d0 2024-03-29T04:08:09.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f25d60640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f1f200b2b60 con 0x7f1f200ca950 2024-03-29T04:08:09.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f25d60640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1f200c6240 con 0x7f1f200b26f0 2024-03-29T04:08:09.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f1effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1f200ca950 0x7f1f200b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:09.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f1f7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1f2015b3d0 0x7f1f200b49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33584/0 (socket says 172.21.15.97:33584) 2024-03-29T04:08:09.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f1f7fe640 1 -- 172.21.15.97:0/3478887867 learned_addr learned my addr 172.21.15.97:0/3478887867 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:09.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f1ffff640 1 --2- 172.21.15.97:0/3478887867 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f200b26f0 0x7f1f2015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:09.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f1effd640 1 -- 172.21.15.97:0/3478887867 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f200b26f0 msgr2=0x7f1f2015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:09.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f1effd640 1 --2- 172.21.15.97:0/3478887867 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f200b26f0 0x7f1f2015be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f1effd640 1 -- 172.21.15.97:0/3478887867 >> v1:172.21.15.97:6789/0 conn(0x7f1f2015b3d0 legacy=0x7f1f200b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:09.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f1effd640 1 -- 172.21.15.97:0/3478887867 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f200c6470 con 0x7f1f200ca950 2024-03-29T04:08:09.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.638+0000 7f1f1e7fc640 1 -- 172.21.15.97:0/3478887867 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1419120341 0 0) 0x7f1f200cade0 con 0x7f1f2015b3d0 2024-03-29T04:08:09.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.639+0000 7f1f1effd640 1 --2- 172.21.15.97:0/3478887867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f200ca950 0x7f1f200b21b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1f140097a0 tx=0x7f1f1402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a8b0a7932c87b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:09.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.639+0000 7f1f1e7fc640 1 -- 172.21.15.97:0/3478887867 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14035020 con 0x7f1f200ca950 2024-03-29T04:08:09.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.639+0000 7f1f1e7fc640 1 -- 172.21.15.97:0/3478887867 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f14031830 con 0x7f1f200ca950 2024-03-29T04:08:09.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.639+0000 7f1f1e7fc640 1 -- 172.21.15.97:0/3478887867 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14031c00 con 0x7f1f200ca950 2024-03-29T04:08:09.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.640+0000 7f1f25d60640 1 -- 172.21.15.97:0/3478887867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f200ca950 msgr2=0x7f1f200b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:09.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.640+0000 7f1f25d60640 1 --2- 172.21.15.97:0/3478887867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f200ca950 0x7f1f200b21b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1f140097a0 tx=0x7f1f1402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.640+0000 7f1f25d60640 1 -- 172.21.15.97:0/3478887867 shutdown_connections 2024-03-29T04:08:09.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.640+0000 7f1f25d60640 1 --2- 172.21.15.97:0/3478887867 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f200b26f0 0x7f1f2015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.640+0000 7f1f25d60640 1 --2- 172.21.15.97:0/3478887867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f200ca950 0x7f1f200b21b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.640+0000 7f1f25d60640 1 -- 172.21.15.97:0/3478887867 >> 172.21.15.97:0/3478887867 conn(0x7f1f200d0450 msgr2=0x7f1f200d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:09.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.640+0000 7f1f25d60640 1 -- 172.21.15.97:0/3478887867 shutdown_connections 2024-03-29T04:08:09.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.640+0000 7f1f25d60640 1 -- 172.21.15.97:0/3478887867 wait complete. 2024-03-29T04:08:09.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.641+0000 7f1f25d60640 1 Processor -- start 2024-03-29T04:08:09.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.641+0000 7f1f25d60640 1 -- start start 2024-03-29T04:08:09.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f25d60640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f200b26f0 0x7f1f200c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:09.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f25d60640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f200ca950 0x7f1f2014b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:09.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f25d60640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f2015b3d0 0x7f1f2014bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:09.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f25d60640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1f200c6a30 con 0x7f1f2015b3d0 2024-03-29T04:08:09.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f25d60640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1f200c7080 con 0x7f1f200b26f0 2024-03-29T04:08:09.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f25d60640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1f200c6d10 con 0x7f1f200ca950 2024-03-29T04:08:09.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f1ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f2015b3d0 0x7f1f2014bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:09.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f1ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f2015b3d0 0x7f1f2014bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39716/0 (socket says 172.21.15.97:39716) 2024-03-29T04:08:09.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f1ffff640 1 -- 172.21.15.97:0/2916521038 learned_addr learned my addr 172.21.15.97:0/2916521038 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:09.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f1f7fe640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f200b26f0 0x7f1f200c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:09.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f1effd640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f200ca950 0x7f1f2014b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:09.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f1ffff640 1 -- 172.21.15.97:0/2916521038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f200ca950 msgr2=0x7f1f2014b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:09.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f1ffff640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f200ca950 0x7f1f2014b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f1ffff640 1 -- 172.21.15.97:0/2916521038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f200b26f0 msgr2=0x7f1f200c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:09.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f1ffff640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f200b26f0 0x7f1f200c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f1ffff640 1 -- 172.21.15.97:0/2916521038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2014c300 con 0x7f1f2015b3d0 2024-03-29T04:08:09.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.642+0000 7f1f1ffff640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f2015b3d0 0x7f1f2014bc60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800efc0 tx=0x7f1f1800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:09.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.643+0000 7f1f07fff640 1 -- 172.21.15.97:0/2916521038 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18018020 con 0x7f1f2015b3d0 2024-03-29T04:08:09.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.643+0000 7f1f07fff640 1 -- 172.21.15.97:0/2916521038 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f18014480 con 0x7f1f2015b3d0 2024-03-29T04:08:09.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.643+0000 7f1f25d60640 1 -- 172.21.15.97:0/2916521038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2014c5e0 con 0x7f1f2015b3d0 2024-03-29T04:08:09.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.643+0000 7f1f07fff640 1 -- 172.21.15.97:0/2916521038 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18014870 con 0x7f1f2015b3d0 2024-03-29T04:08:09.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.643+0000 7f1f25d60640 1 -- 172.21.15.97:0/2916521038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f20149a10 con 0x7f1f2015b3d0 2024-03-29T04:08:09.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.644+0000 7f1f25d60640 1 -- 172.21.15.97:0/2916521038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f2009e520 con 0x7f1f2015b3d0 2024-03-29T04:08:09.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.644+0000 7f1f07fff640 1 -- 172.21.15.97:0/2916521038 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1f1801b020 con 0x7f1f2015b3d0 2024-03-29T04:08:09.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.644+0000 7f1f07fff640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1f0c06c560 0x7f1f0c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:09.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.645+0000 7f1f07fff640 1 -- 172.21.15.97:0/2916521038 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1f1808c0f0 con 0x7f1f2015b3d0 2024-03-29T04:08:09.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.645+0000 7f1f1f7fe640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1f0c06c560 0x7f1f0c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:09.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.645+0000 7f1f1f7fe640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1f0c06c560 0x7f1f0c06ea20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1f10000f60 tx=0x7f1f10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:09.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.649+0000 7f1f07fff640 1 -- 172.21.15.97:0/2916521038 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1f18022030 con 0x7f1f2015b3d0 2024-03-29T04:08:09.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.825+0000 7f1f25d60640 1 -- 172.21.15.97:0/2916521038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1f2004f9d0 con 0x7f1f2015b3d0 2024-03-29T04:08:09.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.828+0000 7f1f07fff640 1 -- 172.21.15.97:0/2916521038 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f1f18010040 con 0x7f1f2015b3d0 2024-03-29T04:08:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.830+0000 7f1f25d60640 1 -- 172.21.15.97:0/2916521038 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1f0c06c560 msgr2=0x7f1f0c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.830+0000 7f1f25d60640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1f0c06c560 0x7f1f0c06ea20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1f10000f60 tx=0x7f1f10009040 comp rx=0 tx=0).stop 2024-03-29T04:08:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.830+0000 7f1f25d60640 1 -- 172.21.15.97:0/2916521038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f2015b3d0 msgr2=0x7f1f2014bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.830+0000 7f1f25d60640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f2015b3d0 0x7f1f2014bc60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800efc0 tx=0x7f1f1800c490 comp rx=0 tx=0).stop 2024-03-29T04:08:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.830+0000 7f1f25d60640 1 -- 172.21.15.97:0/2916521038 shutdown_connections 2024-03-29T04:08:09.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.830+0000 7f1f25d60640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1f0c06c560 0x7f1f0c06ea20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.830+0000 7f1f25d60640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f2015b3d0 0x7f1f2014bc60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.830+0000 7f1f25d60640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f200ca950 0x7f1f2014b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.830+0000 7f1f25d60640 1 --2- 172.21.15.97:0/2916521038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f200b26f0 0x7f1f200c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.830+0000 7f1f25d60640 1 -- 172.21.15.97:0/2916521038 >> 172.21.15.97:0/2916521038 conn(0x7f1f200d0450 msgr2=0x7f1f200d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.830+0000 7f1f25d60640 1 -- 172.21.15.97:0/2916521038 shutdown_connections 2024-03-29T04:08:09.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:09.830+0000 7f1f25d60640 1 -- 172.21.15.97:0/2916521038 wait complete. 2024-03-29T04:08:09.821 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T04:08:09.821 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:08:09.868 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:08:09.911 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T04:08:09.911 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:08:09.911 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:09.927 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:08:09.928 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:09.982 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:08:09.982 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:10.037 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.198+0000 7fa9a2e9a640 1 Processor -- start 2024-03-29T04:08:10.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.198+0000 7fa9a2e9a640 1 -- start start 2024-03-29T04:08:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.199+0000 7fa9a2e9a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa99c15ec60 0x7fa99c0ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.199+0000 7fa9a2e9a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa99c0cf7f0 con 0x7fa99c0b4340 2024-03-29T04:08:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.199+0000 7fa9a2e9a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa99c0cea70 con 0x7fa99c15e880 2024-03-29T04:08:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.199+0000 7fa9a2e9a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fa99c0cfe00 con 0x7fa99c15ec60 2024-03-29T04:08:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.199+0000 7fa9a1437640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa99c0b4340 0x7fa99c0d0290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33596/0 (socket says 172.21.15.97:33596) 2024-03-29T04:08:10.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.199+0000 7fa9a1437640 1 -- 172.21.15.97:0/3351079263 learned_addr learned my addr 172.21.15.97:0/3351079263 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.199+0000 7fa99bfff640 1 --2- 172.21.15.97:0/3351079263 >> v2:172.21.15.154:3300/0 conn(0x7fa99c15ec60 0x7fa99c0ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.199+0000 7fa99b7fe640 1 -- 172.21.15.97:0/3351079263 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 415496055 0 0) 0x7fa99c0cf7f0 con 0x7fa99c0b4340 2024-03-29T04:08:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.199+0000 7fa99b7fe640 1 -- 172.21.15.97:0/3351079263 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa98c003520 con 0x7fa99c0b4340 2024-03-29T04:08:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.199+0000 7fa99bfff640 1 -- 172.21.15.97:0/3351079263 >> v1:172.21.15.154:6789/0 conn(0x7fa99c15e880 legacy=0x7fa99c0cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.199+0000 7fa99bfff640 1 -- 172.21.15.97:0/3351079263 >> v1:172.21.15.97:6789/0 conn(0x7fa99c0b4340 legacy=0x7fa99c0d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.199+0000 7fa99bfff640 1 -- 172.21.15.97:0/3351079263 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c0ced50 con 0x7fa99c15ec60 2024-03-29T04:08:10.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.200+0000 7fa99bfff640 1 --2- 172.21.15.97:0/3351079263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa99c15ec60 0x7fa99c0ce450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa9880097a0 tx=0x7fa98802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15f81376cbabdd96 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:10.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.200+0000 7fa99b7fe640 1 -- 172.21.15.97:0/3351079263 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988034070 con 0x7fa99c15ec60 2024-03-29T04:08:10.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.200+0000 7fa99b7fe640 1 -- 172.21.15.97:0/3351079263 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa988030610 con 0x7fa99c15ec60 2024-03-29T04:08:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.200+0000 7fa99b7fe640 1 -- 172.21.15.97:0/3351079263 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9880309e0 con 0x7fa99c15ec60 2024-03-29T04:08:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.201+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/3351079263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa99c15ec60 msgr2=0x7fa99c0ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.201+0000 7fa9a2e9a640 1 --2- 172.21.15.97:0/3351079263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa99c15ec60 0x7fa99c0ce450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa9880097a0 tx=0x7fa98802b960 comp rx=0 tx=0).stop 2024-03-29T04:08:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.201+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/3351079263 shutdown_connections 2024-03-29T04:08:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.201+0000 7fa9a2e9a640 1 --2- 172.21.15.97:0/3351079263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa99c15ec60 0x7fa99c0ce450 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.201+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/3351079263 >> 172.21.15.97:0/3351079263 conn(0x7fa99c0b6860 msgr2=0x7fa99c0b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.201+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/3351079263 shutdown_connections 2024-03-29T04:08:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.201+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/3351079263 wait complete. 2024-03-29T04:08:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.202+0000 7fa9a2e9a640 1 Processor -- start 2024-03-29T04:08:10.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.202+0000 7fa9a2e9a640 1 -- start start 2024-03-29T04:08:10.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa9a2e9a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa99c0b4340 0x7fa99c14d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa9a2e9a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c15e880 0x7fa99c14be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa9a2e9a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa99c15ec60 0x7fa99c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa9a2e9a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c0d1b30 con 0x7fa99c15e880 2024-03-29T04:08:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa9a2e9a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c0d19b0 con 0x7fa99c15ec60 2024-03-29T04:08:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa9a2e9a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa99c169580 con 0x7fa99c0b4340 2024-03-29T04:08:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c15e880 0x7fa99c14be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa99bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c15e880 0x7fa99c14be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39720/0 (socket says 172.21.15.97:39720) 2024-03-29T04:08:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa99bfff640 1 -- 172.21.15.97:0/328448237 learned_addr learned my addr 172.21.15.97:0/328448237 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa9a1437640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa99c15ec60 0x7fa99c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa9a0c36640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa99c0b4340 0x7fa99c14d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa99bfff640 1 -- 172.21.15.97:0/328448237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa99c0b4340 msgr2=0x7fa99c14d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa99bfff640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa99c0b4340 0x7fa99c14d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa99bfff640 1 -- 172.21.15.97:0/328448237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa99c15ec60 msgr2=0x7fa99c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa99bfff640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa99c15ec60 0x7fa99c160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa99bfff640 1 -- 172.21.15.97:0/328448237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c15f4d0 con 0x7fa99c15e880 2024-03-29T04:08:10.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa99bfff640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c15e880 0x7fa99c14be30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa98802bf70 tx=0x7fa9880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:10.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.203+0000 7fa9997fa640 1 -- 172.21.15.97:0/328448237 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988034040 con 0x7fa99c15e880 2024-03-29T04:08:10.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.204+0000 7fa9997fa640 1 -- 172.21.15.97:0/328448237 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa98803aaa0 con 0x7fa99c15e880 2024-03-29T04:08:10.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.204+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/328448237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99c151620 con 0x7fa99c15e880 2024-03-29T04:08:10.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.204+0000 7fa9997fa640 1 -- 172.21.15.97:0/328448237 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9880443f0 con 0x7fa99c15e880 2024-03-29T04:08:10.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.204+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/328448237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99c14dbf0 con 0x7fa99c15e880 2024-03-29T04:08:10.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.205+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/328448237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa99c068640 con 0x7fa99c15e880 2024-03-29T04:08:10.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.205+0000 7fa9997fa640 1 -- 172.21.15.97:0/328448237 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa98804b060 con 0x7fa99c15e880 2024-03-29T04:08:10.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.205+0000 7fa9997fa640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa98006c560 0x7fa98006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.206+0000 7fa9a0c36640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa98006c560 0x7fa98006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.206+0000 7fa9997fa640 1 -- 172.21.15.97:0/328448237 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa9880ad3c0 con 0x7fa99c15e880 2024-03-29T04:08:10.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.206+0000 7fa9a0c36640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa98006c560 0x7fa98006ea20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa990001640 tx=0x7fa990009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:10.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.210+0000 7fa9997fa640 1 -- 172.21.15.97:0/328448237 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa98803f070 con 0x7fa99c15e880 2024-03-29T04:08:10.231 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:08:10.234 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:08:10.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.389+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/328448237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa99c0b6f70 con 0x7fa99c15e880 2024-03-29T04:08:10.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.390+0000 7fa9997fa640 1 -- 172.21.15.97:0/328448237 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fa98807b9e0 con 0x7fa99c15e880 2024-03-29T04:08:10.370 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:10.370 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:10.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.392+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/328448237 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa98006c560 msgr2=0x7fa98006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.392+0000 7fa9a2e9a640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa98006c560 0x7fa98006ea20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa990001640 tx=0x7fa990009040 comp rx=0 tx=0).stop 2024-03-29T04:08:10.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.392+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/328448237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c15e880 msgr2=0x7fa99c14be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.392+0000 7fa9a2e9a640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c15e880 0x7fa99c14be30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa98802bf70 tx=0x7fa9880046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.392+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/328448237 shutdown_connections 2024-03-29T04:08:10.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.392+0000 7fa9a2e9a640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa98006c560 0x7fa98006ea20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.392+0000 7fa9a2e9a640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa99c15ec60 0x7fa99c160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.392+0000 7fa9a2e9a640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99c15e880 0x7fa99c14be30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.392+0000 7fa9a2e9a640 1 --2- 172.21.15.97:0/328448237 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa99c0b4340 0x7fa99c14d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.392+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/328448237 >> 172.21.15.97:0/328448237 conn(0x7fa99c0b6860 msgr2=0x7fa99c06b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:10.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.392+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/328448237 shutdown_connections 2024-03-29T04:08:10.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.392+0000 7fa9a2e9a640 1 -- 172.21.15.97:0/328448237 wait complete. 2024-03-29T04:08:10.383 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:07:38.618704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:07:39.724339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:08:10.383 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:08:10.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.507+0000 7fc97d72f640 1 Processor -- start 2024-03-29T04:08:10.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.508+0000 7fc97d72f640 1 -- start start 2024-03-29T04:08:10.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.508+0000 7fc97d72f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9780cfde0 0x7fc9780d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.508+0000 7fc97d72f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc9780d0f90 0x7fc9780d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.508+0000 7fc97d72f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97815ed00 con 0x7fc97815e920 2024-03-29T04:08:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.508+0000 7fc97d72f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fc9780d21e0 con 0x7fc9780cfde0 2024-03-29T04:08:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.508+0000 7fc97d72f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fc9780d2480 con 0x7fc9780d0f90 2024-03-29T04:08:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.508+0000 7fc9767fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9780cfde0 0x7fc9780d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.508+0000 7fc976ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc97815e920 0x7fc9780d3240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33602/0 (socket says 172.21.15.97:33602) 2024-03-29T04:08:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.508+0000 7fc976ffd640 1 -- 172.21.15.97:0/981408727 learned_addr learned my addr 172.21.15.97:0/981408727 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.509+0000 7fc9777fe640 1 --2- 172.21.15.97:0/981408727 >> v2:172.21.15.154:3300/0 conn(0x7fc9780d0f90 0x7fc9780d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.509+0000 7fc9767fc640 1 -- 172.21.15.97:0/981408727 >> v2:172.21.15.154:3300/0 conn(0x7fc9780d0f90 msgr2=0x7fc9780d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.509+0000 7fc9767fc640 1 --2- 172.21.15.97:0/981408727 >> v2:172.21.15.154:3300/0 conn(0x7fc9780d0f90 0x7fc9780d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.509+0000 7fc9767fc640 1 -- 172.21.15.97:0/981408727 >> v1:172.21.15.97:6789/0 conn(0x7fc97815e920 legacy=0x7fc9780d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.509+0000 7fc9767fc640 1 -- 172.21.15.97:0/981408727 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9780d2760 con 0x7fc9780cfde0 2024-03-29T04:08:10.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.509+0000 7fc975ffb640 1 -- 172.21.15.97:0/981408727 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2005900753 0 0) 0x7fc97815ed00 con 0x7fc97815e920 2024-03-29T04:08:10.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.509+0000 7fc9767fc640 1 --2- 172.21.15.97:0/981408727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9780cfde0 0x7fc9780d0240 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc97808dab0 tx=0x7fc96c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aeecfcbbd5e34b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:10.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.510+0000 7fc975ffb640 1 -- 172.21.15.97:0/981408727 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c035020 con 0x7fc9780cfde0 2024-03-29T04:08:10.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.510+0000 7fc975ffb640 1 -- 172.21.15.97:0/981408727 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc96c031830 con 0x7fc9780cfde0 2024-03-29T04:08:10.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.510+0000 7fc975ffb640 1 -- 172.21.15.97:0/981408727 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c031c00 con 0x7fc9780cfde0 2024-03-29T04:08:10.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.510+0000 7fc97d72f640 1 -- 172.21.15.97:0/981408727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9780cfde0 msgr2=0x7fc9780d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.510+0000 7fc97d72f640 1 --2- 172.21.15.97:0/981408727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9780cfde0 0x7fc9780d0240 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc97808dab0 tx=0x7fc96c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.510+0000 7fc97d72f640 1 -- 172.21.15.97:0/981408727 shutdown_connections 2024-03-29T04:08:10.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.510+0000 7fc97d72f640 1 --2- 172.21.15.97:0/981408727 >> v2:172.21.15.154:3300/0 conn(0x7fc9780d0f90 0x7fc9780d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.510+0000 7fc97d72f640 1 --2- 172.21.15.97:0/981408727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9780cfde0 0x7fc9780d0240 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.510+0000 7fc97d72f640 1 -- 172.21.15.97:0/981408727 >> 172.21.15.97:0/981408727 conn(0x7fc97808bfc0 msgr2=0x7fc9780d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:10.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.510+0000 7fc97d72f640 1 -- 172.21.15.97:0/981408727 shutdown_connections 2024-03-29T04:08:10.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.510+0000 7fc97d72f640 1 -- 172.21.15.97:0/981408727 wait complete. 2024-03-29T04:08:10.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.511+0000 7fc97d72f640 1 Processor -- start 2024-03-29T04:08:10.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.511+0000 7fc97d72f640 1 -- start start 2024-03-29T04:08:10.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.512+0000 7fc97d72f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9780cfde0 0x7fc978150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.512+0000 7fc97d72f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9780d0f90 0x7fc978160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.512+0000 7fc97d72f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97815e920 0x7fc9780cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.512+0000 7fc97d72f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc9780d4ef0 con 0x7fc9780cfde0 2024-03-29T04:08:10.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.512+0000 7fc97d72f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc9780d2d20 con 0x7fc97815e920 2024-03-29T04:08:10.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.512+0000 7fc97d72f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc9780d5070 con 0x7fc9780d0f90 2024-03-29T04:08:10.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.512+0000 7fc976ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9780cfde0 0x7fc978150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.512+0000 7fc976ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9780cfde0 0x7fc978150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34574/0 (socket says 172.21.15.97:34574) 2024-03-29T04:08:10.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.512+0000 7fc976ffd640 1 -- 172.21.15.97:0/1795769792 learned_addr learned my addr 172.21.15.97:0/1795769792 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:10.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.512+0000 7fc9777fe640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97815e920 0x7fc9780cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.512+0000 7fc9767fc640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9780d0f90 0x7fc978160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.513+0000 7fc976ffd640 1 -- 172.21.15.97:0/1795769792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9780d0f90 msgr2=0x7fc978160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.513+0000 7fc976ffd640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9780d0f90 0x7fc978160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.513+0000 7fc976ffd640 1 -- 172.21.15.97:0/1795769792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97815e920 msgr2=0x7fc9780cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.513+0000 7fc976ffd640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97815e920 0x7fc9780cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.513+0000 7fc976ffd640 1 -- 172.21.15.97:0/1795769792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97815f090 con 0x7fc9780cfde0 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.513+0000 7fc9767fc640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9780d0f90 0x7fc978160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.513+0000 7fc976ffd640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9780cfde0 0x7fc978150740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc96800c8c0 tx=0x7fc96800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.513+0000 7fc95f7fe640 1 -- 172.21.15.97:0/1795769792 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc968018070 con 0x7fc9780cfde0 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.513+0000 7fc95f7fe640 1 -- 172.21.15.97:0/1795769792 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc96800a640 con 0x7fc9780cfde0 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.513+0000 7fc97d72f640 1 -- 172.21.15.97:0/1795769792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97815f370 con 0x7fc9780cfde0 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.513+0000 7fc95f7fe640 1 -- 172.21.15.97:0/1795769792 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96800aa30 con 0x7fc9780cfde0 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.513+0000 7fc97d72f640 1 -- 172.21.15.97:0/1795769792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97815f930 con 0x7fc9780cfde0 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.514+0000 7fc97d72f640 1 -- 172.21.15.97:0/1795769792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9780a2560 con 0x7fc9780cfde0 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.514+0000 7fc95f7fe640 1 -- 172.21.15.97:0/1795769792 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc96801b070 con 0x7fc9780cfde0 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.515+0000 7fc95f7fe640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc96406c560 0x7fc96406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.515+0000 7fc95f7fe640 1 -- 172.21.15.97:0/1795769792 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc96808ad10 con 0x7fc9780cfde0 2024-03-29T04:08:10.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.515+0000 7fc9767fc640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc96406c560 0x7fc96406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.515+0000 7fc9767fc640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc96406c560 0x7fc96406ea20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc96c036000 tx=0x7fc96c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:10.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.519+0000 7fc95f7fe640 1 -- 172.21.15.97:0/1795769792 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc968058ce0 con 0x7fc9780cfde0 2024-03-29T04:08:10.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.696+0000 7fc97d72f640 1 -- 172.21.15.97:0/1795769792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc97804f9d0 con 0x7fc9780cfde0 2024-03-29T04:08:10.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.696+0000 7fc95f7fe640 1 -- 172.21.15.97:0/1795769792 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fc96805cfe0 con 0x7fc9780cfde0 2024-03-29T04:08:10.676 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:08:10.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.698+0000 7fc97d72f640 1 -- 172.21.15.97:0/1795769792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc96406c560 msgr2=0x7fc96406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.698+0000 7fc97d72f640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc96406c560 0x7fc96406ea20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc96c036000 tx=0x7fc96c03c040 comp rx=0 tx=0).stop 2024-03-29T04:08:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.698+0000 7fc97d72f640 1 -- 172.21.15.97:0/1795769792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9780cfde0 msgr2=0x7fc978150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.698+0000 7fc97d72f640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9780cfde0 0x7fc978150740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc96800c8c0 tx=0x7fc96800cd80 comp rx=0 tx=0).stop 2024-03-29T04:08:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.698+0000 7fc97d72f640 1 -- 172.21.15.97:0/1795769792 shutdown_connections 2024-03-29T04:08:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.698+0000 7fc97d72f640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc96406c560 0x7fc96406ea20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.698+0000 7fc97d72f640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc97815e920 0x7fc9780cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.699+0000 7fc97d72f640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9780d0f90 0x7fc978160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.699+0000 7fc97d72f640 1 --2- 172.21.15.97:0/1795769792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9780cfde0 0x7fc978150740 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.699+0000 7fc97d72f640 1 -- 172.21.15.97:0/1795769792 >> 172.21.15.97:0/1795769792 conn(0x7fc97808bfc0 msgr2=0x7fc9780d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.699+0000 7fc97d72f640 1 -- 172.21.15.97:0/1795769792 shutdown_connections 2024-03-29T04:08:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.699+0000 7fc97d72f640 1 -- 172.21.15.97:0/1795769792 wait complete. 2024-03-29T04:08:10.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:08:10.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.812+0000 7f1e97972640 1 Processor -- start 2024-03-29T04:08:10.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.812+0000 7f1e97972640 1 -- start start 2024-03-29T04:08:10.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.813+0000 7f1e97972640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1e9015e830 0x7f1e9006c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.813+0000 7f1e97972640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e9006ca60 0x7f1e9006e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.813+0000 7f1e97972640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e9006ed90 con 0x7f1e9006e630 2024-03-29T04:08:10.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.813+0000 7f1e97972640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f1e9006cec0 con 0x7f1e9006ca60 2024-03-29T04:08:10.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.813+0000 7f1e97972640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f1e9006f070 con 0x7f1e9015e830 2024-03-29T04:08:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.813+0000 7f1e94f0d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e9006ca60 0x7f1e9006e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.813+0000 7f1e94f0d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e9006ca60 0x7f1e9006e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34580/0 (socket says 172.21.15.97:34580) 2024-03-29T04:08:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.813+0000 7f1e94f0d640 1 -- 172.21.15.97:0/217905439 learned_addr learned my addr 172.21.15.97:0/217905439 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.813+0000 7f1e9570e640 1 --2- 172.21.15.97:0/217905439 >> v2:172.21.15.154:3300/0 conn(0x7f1e9015e830 0x7f1e9006c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.814+0000 7f1e94f0d640 1 -- 172.21.15.97:0/217905439 >> v2:172.21.15.154:3300/0 conn(0x7f1e9015e830 msgr2=0x7f1e9006c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.814+0000 7f1e94f0d640 1 --2- 172.21.15.97:0/217905439 >> v2:172.21.15.154:3300/0 conn(0x7f1e9015e830 0x7f1e9006c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.814+0000 7f1e94f0d640 1 -- 172.21.15.97:0/217905439 >> v1:172.21.15.154:6789/0 conn(0x7f1e9006e630 legacy=0x7f1e90162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.814+0000 7f1e94f0d640 1 -- 172.21.15.97:0/217905439 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e9006f350 con 0x7f1e9006ca60 2024-03-29T04:08:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.814+0000 7f1e94f0d640 1 --2- 172.21.15.97:0/217905439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9006ca60 0x7f1e9006e0f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1e90073d90 tx=0x7f1e8002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91bdfa96585c246 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.814+0000 7f1e87fff640 1 -- 172.21.15.97:0/217905439 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80034070 con 0x7f1e9006ca60 2024-03-29T04:08:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.814+0000 7f1e87fff640 1 -- 172.21.15.97:0/217905439 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e80030860 con 0x7f1e9006ca60 2024-03-29T04:08:10.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.815+0000 7f1e87fff640 1 -- 172.21.15.97:0/217905439 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80030c30 con 0x7f1e9006ca60 2024-03-29T04:08:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.815+0000 7f1e97972640 1 -- 172.21.15.97:0/217905439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9006ca60 msgr2=0x7f1e9006e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.815+0000 7f1e97972640 1 --2- 172.21.15.97:0/217905439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9006ca60 0x7f1e9006e0f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1e90073d90 tx=0x7f1e8002b9a0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.815+0000 7f1e97972640 1 -- 172.21.15.97:0/217905439 shutdown_connections 2024-03-29T04:08:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.815+0000 7f1e97972640 1 --2- 172.21.15.97:0/217905439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9006ca60 0x7f1e9006e0f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.815+0000 7f1e97972640 1 --2- 172.21.15.97:0/217905439 >> v2:172.21.15.154:3300/0 conn(0x7f1e9015e830 0x7f1e9006c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.815+0000 7f1e97972640 1 -- 172.21.15.97:0/217905439 >> 172.21.15.97:0/217905439 conn(0x7f1e90072090 msgr2=0x7f1e900724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.815+0000 7f1e97972640 1 -- 172.21.15.97:0/217905439 shutdown_connections 2024-03-29T04:08:10.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.815+0000 7f1e97972640 1 -- 172.21.15.97:0/217905439 wait complete. 2024-03-29T04:08:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.816+0000 7f1e97972640 1 Processor -- start 2024-03-29T04:08:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.816+0000 7f1e97972640 1 -- start start 2024-03-29T04:08:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e97972640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e9006ca60 0x7f1e90160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e97972640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9006e630 0x7f1e900cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e97972640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e9015e830 0x7f1e900cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e97972640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9006eaa0 con 0x7f1e9006e630 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e97972640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9006f910 con 0x7f1e9006ca60 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e97972640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1e900ae9f0 con 0x7f1e9015e830 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e94f0d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9006e630 0x7f1e900cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e94f0d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9006e630 0x7f1e900cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34582/0 (socket says 172.21.15.97:34582) 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e94f0d640 1 -- 172.21.15.97:0/913633152 learned_addr learned my addr 172.21.15.97:0/913633152 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e9570e640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e9006ca60 0x7f1e90160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e95f0f640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e9015e830 0x7f1e900cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e94f0d640 1 -- 172.21.15.97:0/913633152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e9015e830 msgr2=0x7f1e900cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e94f0d640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e9015e830 0x7f1e900cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e94f0d640 1 -- 172.21.15.97:0/913633152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e9006ca60 msgr2=0x7f1e90160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e94f0d640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e9006ca60 0x7f1e90160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e94f0d640 1 -- 172.21.15.97:0/913633152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e90160fa0 con 0x7f1e9006e630 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.817+0000 7f1e94f0d640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9006e630 0x7f1e900cf730 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1e8002b9e0 tx=0x7f1e80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.818+0000 7f1e85ffb640 1 -- 172.21.15.97:0/913633152 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80048070 con 0x7f1e9006e630 2024-03-29T04:08:10.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.818+0000 7f1e85ffb640 1 -- 172.21.15.97:0/913633152 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e80030660 con 0x7f1e9006e630 2024-03-29T04:08:10.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.818+0000 7f1e97972640 1 -- 172.21.15.97:0/913633152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e90161280 con 0x7f1e9006e630 2024-03-29T04:08:10.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.818+0000 7f1e85ffb640 1 -- 172.21.15.97:0/913633152 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80034040 con 0x7f1e9006e630 2024-03-29T04:08:10.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.818+0000 7f1e97972640 1 -- 172.21.15.97:0/913633152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e90161840 con 0x7f1e9006e630 2024-03-29T04:08:10.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.819+0000 7f1e97972640 1 -- 172.21.15.97:0/913633152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e90162200 con 0x7f1e9006e630 2024-03-29T04:08:10.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.819+0000 7f1e85ffb640 1 -- 172.21.15.97:0/913633152 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1e8004b070 con 0x7f1e9006e630 2024-03-29T04:08:10.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.819+0000 7f1e85ffb640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1e7406c530 0x7f1e7406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:10.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.820+0000 7f1e85ffb640 1 -- 172.21.15.97:0/913633152 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1e800ad500 con 0x7f1e9006e630 2024-03-29T04:08:10.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.820+0000 7f1e9570e640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1e7406c530 0x7f1e7406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:10.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.820+0000 7f1e9570e640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1e7406c530 0x7f1e7406e9f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c011340 tx=0x7f1e8c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:10.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.824+0000 7f1e85ffb640 1 -- 172.21.15.97:0/913633152 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1e8007b560 con 0x7f1e9006e630 2024-03-29T04:08:10.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.998+0000 7f1e97972640 1 -- 172.21.15.97:0/913633152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1e90067550 con 0x7f1e9006e630 2024-03-29T04:08:10.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:10.999+0000 7f1e85ffb640 1 -- 172.21.15.97:0/913633152 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f1e8007f860 con 0x7f1e9006e630 2024-03-29T04:08:10.979 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:08:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.001+0000 7f1e97972640 1 -- 172.21.15.97:0/913633152 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1e7406c530 msgr2=0x7f1e7406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.001+0000 7f1e97972640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1e7406c530 0x7f1e7406e9f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c011340 tx=0x7f1e8c00b040 comp rx=0 tx=0).stop 2024-03-29T04:08:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.001+0000 7f1e97972640 1 -- 172.21.15.97:0/913633152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9006e630 msgr2=0x7f1e900cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.001+0000 7f1e97972640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9006e630 0x7f1e900cf730 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1e8002b9e0 tx=0x7f1e80004730 comp rx=0 tx=0).stop 2024-03-29T04:08:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.001+0000 7f1e97972640 1 -- 172.21.15.97:0/913633152 shutdown_connections 2024-03-29T04:08:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.001+0000 7f1e97972640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1e7406c530 0x7f1e7406e9f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.001+0000 7f1e97972640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1e9015e830 0x7f1e900cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.001+0000 7f1e97972640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9006e630 0x7f1e900cf730 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.001+0000 7f1e97972640 1 --2- 172.21.15.97:0/913633152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e9006ca60 0x7f1e90160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:10.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.001+0000 7f1e97972640 1 -- 172.21.15.97:0/913633152 >> 172.21.15.97:0/913633152 conn(0x7f1e90072090 msgr2=0x7f1e90154410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:10.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.001+0000 7f1e97972640 1 -- 172.21.15.97:0/913633152 shutdown_connections 2024-03-29T04:08:10.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.001+0000 7f1e97972640 1 -- 172.21.15.97:0/913633152 wait complete. 2024-03-29T04:08:10.991 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:08:11.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.115+0000 7f32f5649640 1 Processor -- start 2024-03-29T04:08:11.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.115+0000 7f32f5649640 1 -- start start 2024-03-29T04:08:11.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32f5649640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f32f015e920 0x7f32f00d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32f5649640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f32f00d16a0 0x7f32f00d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32f5649640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f00d3780 con 0x7f32f00cfde0 2024-03-29T04:08:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32f5649640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f32f015ed00 con 0x7f32f00d16a0 2024-03-29T04:08:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32f5649640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f32f00d21e0 con 0x7f32f015e920 2024-03-29T04:08:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32ef7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f32f00d16a0 0x7f32f00d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32ee7fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f32f00cfde0 0x7f32f00d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35368/0 (socket says 172.21.15.97:35368) 2024-03-29T04:08:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32ef7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f32f00d16a0 0x7f32f00d0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34584/0 (socket says 172.21.15.97:34584) 2024-03-29T04:08:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32ee7fc640 1 -- 172.21.15.97:0/668845410 learned_addr learned my addr 172.21.15.97:0/668845410 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32eeffd640 1 --2- 172.21.15.97:0/668845410 >> v2:172.21.15.154:3300/0 conn(0x7f32f015e920 0x7f32f00d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32ef7fe640 1 -- 172.21.15.97:0/668845410 >> v2:172.21.15.154:3300/0 conn(0x7f32f015e920 msgr2=0x7f32f00d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32ef7fe640 1 --2- 172.21.15.97:0/668845410 >> v2:172.21.15.154:3300/0 conn(0x7f32f015e920 0x7f32f00d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32ef7fe640 1 -- 172.21.15.97:0/668845410 >> v1:172.21.15.97:6789/0 conn(0x7f32f00cfde0 legacy=0x7f32f00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32ef7fe640 1 -- 172.21.15.97:0/668845410 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f00d2480 con 0x7f32f00d16a0 2024-03-29T04:08:11.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.116+0000 7f32edffb640 1 -- 172.21.15.97:0/668845410 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1776794818 0 0) 0x7f32e4001110 con 0x7f32f00cfde0 2024-03-29T04:08:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.117+0000 7f32ef7fe640 1 --2- 172.21.15.97:0/668845410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f00d16a0 0x7f32f00d0240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32f008e880 tx=0x7f32e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3206e06ffada7e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.117+0000 7f32edffb640 1 -- 172.21.15.97:0/668845410 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8039070 con 0x7f32f00d16a0 2024-03-29T04:08:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.117+0000 7f32edffb640 1 -- 172.21.15.97:0/668845410 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e802cc80 con 0x7f32f00d16a0 2024-03-29T04:08:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.117+0000 7f32edffb640 1 -- 172.21.15.97:0/668845410 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8034660 con 0x7f32f00d16a0 2024-03-29T04:08:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.118+0000 7f32f5649640 1 -- 172.21.15.97:0/668845410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f00d16a0 msgr2=0x7f32f00d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.118+0000 7f32f5649640 1 --2- 172.21.15.97:0/668845410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f00d16a0 0x7f32f00d0240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32f008e880 tx=0x7f32e802bed0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.118+0000 7f32f5649640 1 -- 172.21.15.97:0/668845410 shutdown_connections 2024-03-29T04:08:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.118+0000 7f32f5649640 1 --2- 172.21.15.97:0/668845410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f00d16a0 0x7f32f00d0240 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.118+0000 7f32f5649640 1 --2- 172.21.15.97:0/668845410 >> v2:172.21.15.154:3300/0 conn(0x7f32f015e920 0x7f32f00d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.118+0000 7f32f5649640 1 -- 172.21.15.97:0/668845410 >> 172.21.15.97:0/668845410 conn(0x7f32f008bfc0 msgr2=0x7f32f00d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.118+0000 7f32f5649640 1 -- 172.21.15.97:0/668845410 shutdown_connections 2024-03-29T04:08:11.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.118+0000 7f32f5649640 1 -- 172.21.15.97:0/668845410 wait complete. 2024-03-29T04:08:11.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.119+0000 7f32f5649640 1 Processor -- start 2024-03-29T04:08:11.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.119+0000 7f32f5649640 1 -- start start 2024-03-29T04:08:11.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32f5649640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32f00cfde0 0x7f32f00cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32f5649640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f00d16a0 0x7f32f0157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32f5649640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32f015e920 0x7f32f01ea660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32f5649640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f32f00d2d20 con 0x7f32f00d16a0 2024-03-29T04:08:11.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32f5649640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f32f00d2a40 con 0x7f32f00cfde0 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32f5649640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f32f00d4ef0 con 0x7f32f015e920 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32ee7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f00d16a0 0x7f32f0157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32ee7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f00d16a0 0x7f32f0157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34592/0 (socket says 172.21.15.97:34592) 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32ee7fc640 1 -- 172.21.15.97:0/499250970 learned_addr learned my addr 172.21.15.97:0/499250970 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32eeffd640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32f00cfde0 0x7f32f00cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32ef7fe640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32f015e920 0x7f32f01ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32ee7fc640 1 -- 172.21.15.97:0/499250970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32f015e920 msgr2=0x7f32f01ea660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32ee7fc640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32f015e920 0x7f32f01ea660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32ee7fc640 1 -- 172.21.15.97:0/499250970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32f00cfde0 msgr2=0x7f32f00cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32ee7fc640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32f00cfde0 0x7f32f00cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32ee7fc640 1 -- 172.21.15.97:0/499250970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f00cc490 con 0x7f32f00d16a0 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32ee7fc640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f00d16a0 0x7f32f0157720 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32e400c430 tx=0x7f32e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32d77fe640 1 -- 172.21.15.97:0/499250970 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e40074e0 con 0x7f32f00d16a0 2024-03-29T04:08:11.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32d77fe640 1 -- 172.21.15.97:0/499250970 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e40047e0 con 0x7f32f00d16a0 2024-03-29T04:08:11.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32f5649640 1 -- 172.21.15.97:0/499250970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f00cc770 con 0x7f32f00d16a0 2024-03-29T04:08:11.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.120+0000 7f32d77fe640 1 -- 172.21.15.97:0/499250970 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4004bd0 con 0x7f32f00d16a0 2024-03-29T04:08:11.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.121+0000 7f32f5649640 1 -- 172.21.15.97:0/499250970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f00ccd30 con 0x7f32f00d16a0 2024-03-29T04:08:11.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.121+0000 7f32f5649640 1 -- 172.21.15.97:0/499250970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f00cd210 con 0x7f32f00d16a0 2024-03-29T04:08:11.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.122+0000 7f32d77fe640 1 -- 172.21.15.97:0/499250970 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f32e4011070 con 0x7f32f00d16a0 2024-03-29T04:08:11.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.122+0000 7f32d77fe640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f32dc06c530 0x7f32dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.122+0000 7f32d77fe640 1 -- 172.21.15.97:0/499250970 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f32e408b290 con 0x7f32f00d16a0 2024-03-29T04:08:11.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.122+0000 7f32eeffd640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f32dc06c530 0x7f32dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.123+0000 7f32eeffd640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f32dc06c530 0x7f32dc06e9f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009b40 tx=0x7f32e0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:11.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.126+0000 7f32d77fe640 1 -- 172.21.15.97:0/499250970 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f32f00cd210 con 0x7f32f00d16a0 2024-03-29T04:08:11.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.299+0000 7f32f5649640 1 -- 172.21.15.97:0/499250970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f32f004f9d0 con 0x7f32f00d16a0 2024-03-29T04:08:11.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.300+0000 7f32d77fe640 1 -- 172.21.15.97:0/499250970 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f32f00d3780 con 0x7f32f00d16a0 2024-03-29T04:08:11.280 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:08:11.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.302+0000 7f32f5649640 1 -- 172.21.15.97:0/499250970 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f32dc06c530 msgr2=0x7f32dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.302+0000 7f32f5649640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f32dc06c530 0x7f32dc06e9f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009b40 tx=0x7f32e0009190 comp rx=0 tx=0).stop 2024-03-29T04:08:11.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.302+0000 7f32f5649640 1 -- 172.21.15.97:0/499250970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f00d16a0 msgr2=0x7f32f0157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.302+0000 7f32f5649640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f00d16a0 0x7f32f0157720 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32e400c430 tx=0x7f32e400c8f0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.302+0000 7f32f5649640 1 -- 172.21.15.97:0/499250970 shutdown_connections 2024-03-29T04:08:11.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.302+0000 7f32f5649640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f32dc06c530 0x7f32dc06e9f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.302+0000 7f32f5649640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32f015e920 0x7f32f01ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.302+0000 7f32f5649640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f00d16a0 0x7f32f0157720 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.302+0000 7f32f5649640 1 --2- 172.21.15.97:0/499250970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32f00cfde0 0x7f32f00cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.302+0000 7f32f5649640 1 -- 172.21.15.97:0/499250970 >> 172.21.15.97:0/499250970 conn(0x7f32f008bfc0 msgr2=0x7f32f00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:11.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.302+0000 7f32f5649640 1 -- 172.21.15.97:0/499250970 shutdown_connections 2024-03-29T04:08:11.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.302+0000 7f32f5649640 1 -- 172.21.15.97:0/499250970 wait complete. 2024-03-29T04:08:11.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:08:11.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.414+0000 7fc95f760640 1 Processor -- start 2024-03-29T04:08:11.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.414+0000 7fc95f760640 1 -- start start 2024-03-29T04:08:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.415+0000 7fc95f760640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc95815e780 0x7fc9580cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.415+0000 7fc95f760640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc9580ba000 0x7fc9580d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.415+0000 7fc95f760640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9580cfe70 con 0x7fc95815eb60 2024-03-29T04:08:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.415+0000 7fc95f760640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fc9580b7f50 con 0x7fc95815e780 2024-03-29T04:08:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.415+0000 7fc95f760640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fc9580b8230 con 0x7fc9580ba000 2024-03-29T04:08:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.415+0000 7fc95d4fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc95815e780 0x7fc9580cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.415+0000 7fc95d4fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc95815e780 0x7fc9580cebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34602/0 (socket says 172.21.15.97:34602) 2024-03-29T04:08:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.415+0000 7fc95d4fc640 1 -- 172.21.15.97:0/2002723740 learned_addr learned my addr 172.21.15.97:0/2002723740 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.415+0000 7fc95dcfd640 1 --2- 172.21.15.97:0/2002723740 >> v2:172.21.15.154:3300/0 conn(0x7fc9580ba000 0x7fc9580d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.416+0000 7fc95d4fc640 1 -- 172.21.15.97:0/2002723740 >> v2:172.21.15.154:3300/0 conn(0x7fc9580ba000 msgr2=0x7fc9580d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.416+0000 7fc95d4fc640 1 --2- 172.21.15.97:0/2002723740 >> v2:172.21.15.154:3300/0 conn(0x7fc9580ba000 0x7fc9580d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.416+0000 7fc95d4fc640 1 -- 172.21.15.97:0/2002723740 >> v1:172.21.15.154:6789/0 conn(0x7fc95815eb60 legacy=0x7fc9580cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.416+0000 7fc95d4fc640 1 -- 172.21.15.97:0/2002723740 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9580b8510 con 0x7fc95815e780 2024-03-29T04:08:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.416+0000 7fc95d4fc640 1 --2- 172.21.15.97:0/2002723740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc95815e780 0x7fc9580cebc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc95806c3a0 tx=0x7fc95402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cdddec9015eabf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.417+0000 7fc94ffff640 1 -- 172.21.15.97:0/2002723740 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95402cb20 con 0x7fc95815e780 2024-03-29T04:08:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.417+0000 7fc94ffff640 1 -- 172.21.15.97:0/2002723740 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95402ccc0 con 0x7fc95815e780 2024-03-29T04:08:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.417+0000 7fc94ffff640 1 -- 172.21.15.97:0/2002723740 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954034610 con 0x7fc95815e780 2024-03-29T04:08:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.417+0000 7fc95f760640 1 -- 172.21.15.97:0/2002723740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc95815e780 msgr2=0x7fc9580cebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.417+0000 7fc95f760640 1 --2- 172.21.15.97:0/2002723740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc95815e780 0x7fc9580cebc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc95806c3a0 tx=0x7fc95402bf80 comp rx=0 tx=0).stop 2024-03-29T04:08:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.417+0000 7fc95f760640 1 -- 172.21.15.97:0/2002723740 shutdown_connections 2024-03-29T04:08:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.417+0000 7fc95f760640 1 --2- 172.21.15.97:0/2002723740 >> v2:172.21.15.154:3300/0 conn(0x7fc9580ba000 0x7fc9580d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.417+0000 7fc95f760640 1 --2- 172.21.15.97:0/2002723740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc95815e780 0x7fc9580cebc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.417+0000 7fc95f760640 1 -- 172.21.15.97:0/2002723740 >> 172.21.15.97:0/2002723740 conn(0x7fc95808bfc0 msgr2=0x7fc9580d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.417+0000 7fc95f760640 1 -- 172.21.15.97:0/2002723740 shutdown_connections 2024-03-29T04:08:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.417+0000 7fc95f760640 1 -- 172.21.15.97:0/2002723740 wait complete. 2024-03-29T04:08:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.418+0000 7fc95f760640 1 Processor -- start 2024-03-29T04:08:11.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.418+0000 7fc95f760640 1 -- start start 2024-03-29T04:08:11.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95f760640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9580ba000 0x7fc958153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95f760640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc95815e780 0x7fc95815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95f760640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc95815c7f0 0x7fc958158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95f760640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc9580cf810 con 0x7fc95815e780 2024-03-29T04:08:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95f760640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc9580b8ad0 con 0x7fc95815c7f0 2024-03-29T04:08:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95f760640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc9580cf990 con 0x7fc9580ba000 2024-03-29T04:08:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95ccfb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc95815e780 0x7fc95815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95ccfb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc95815e780 0x7fc95815c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34608/0 (socket says 172.21.15.97:34608) 2024-03-29T04:08:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95ccfb640 1 -- 172.21.15.97:0/1162013493 learned_addr learned my addr 172.21.15.97:0/1162013493 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95d4fc640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9580ba000 0x7fc958153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95dcfd640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc95815c7f0 0x7fc958158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95ccfb640 1 -- 172.21.15.97:0/1162013493 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9580ba000 msgr2=0x7fc958153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95ccfb640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9580ba000 0x7fc958153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95ccfb640 1 -- 172.21.15.97:0/1162013493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc95815c7f0 msgr2=0x7fc958158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95ccfb640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc95815c7f0 0x7fc958158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.419+0000 7fc95ccfb640 1 -- 172.21.15.97:0/1162013493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc958153f60 con 0x7fc95815e780 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.420+0000 7fc95ccfb640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc95815e780 0x7fc95815c2b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc94800c3c0 tx=0x7fc94800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.420+0000 7fc94dffb640 1 -- 172.21.15.97:0/1162013493 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480052d0 con 0x7fc95815e780 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.420+0000 7fc94dffb640 1 -- 172.21.15.97:0/1162013493 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc948005490 con 0x7fc95815e780 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.420+0000 7fc95f760640 1 -- 172.21.15.97:0/1162013493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc958159150 con 0x7fc95815e780 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.420+0000 7fc94dffb640 1 -- 172.21.15.97:0/1162013493 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948007260 con 0x7fc95815e780 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.420+0000 7fc95f760640 1 -- 172.21.15.97:0/1162013493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc958159710 con 0x7fc95815e780 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.421+0000 7fc95f760640 1 -- 172.21.15.97:0/1162013493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc95815cc00 con 0x7fc95815e780 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.421+0000 7fc94dffb640 1 -- 172.21.15.97:0/1162013493 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc948007400 con 0x7fc95815e780 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.422+0000 7fc94dffb640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc93c06c560 0x7fc93c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.422+0000 7fc94dffb640 1 -- 172.21.15.97:0/1162013493 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc94808aa30 con 0x7fc95815e780 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.422+0000 7fc95d4fc640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc93c06c560 0x7fc93c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.422+0000 7fc95d4fc640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc93c06c560 0x7fc93c06ea20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc954035000 tx=0x7fc954030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:11.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.426+0000 7fc94dffb640 1 -- 172.21.15.97:0/1162013493 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc95815cc00 con 0x7fc95815e780 2024-03-29T04:08:11.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.615+0000 7fc95f760640 1 -- 172.21.15.97:0/1162013493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc9580b65b0 con 0x7fc95815e780 2024-03-29T04:08:11.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.616+0000 7fc94dffb640 1 -- 172.21.15.97:0/1162013493 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fc948058a90 con 0x7fc95815e780 2024-03-29T04:08:11.596 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-03-29T04:08:11.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.618+0000 7fc95f760640 1 -- 172.21.15.97:0/1162013493 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc93c06c560 msgr2=0x7fc93c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.618+0000 7fc95f760640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc93c06c560 0x7fc93c06ea20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc954035000 tx=0x7fc954030000 comp rx=0 tx=0).stop 2024-03-29T04:08:11.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.618+0000 7fc95f760640 1 -- 172.21.15.97:0/1162013493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc95815e780 msgr2=0x7fc95815c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.618+0000 7fc95f760640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc95815e780 0x7fc95815c2b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc94800c3c0 tx=0x7fc94800c880 comp rx=0 tx=0).stop 2024-03-29T04:08:11.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.618+0000 7fc95f760640 1 -- 172.21.15.97:0/1162013493 shutdown_connections 2024-03-29T04:08:11.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.618+0000 7fc95f760640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc93c06c560 0x7fc93c06ea20 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc954035000 tx=0x7fc954030000 comp rx=0 tx=0).stop 2024-03-29T04:08:11.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.618+0000 7fc95f760640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc95815c7f0 0x7fc958158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.618+0000 7fc95f760640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc95815e780 0x7fc95815c2b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.618+0000 7fc95f760640 1 --2- 172.21.15.97:0/1162013493 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc9580ba000 0x7fc958153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.618+0000 7fc95f760640 1 -- 172.21.15.97:0/1162013493 >> 172.21.15.97:0/1162013493 conn(0x7fc95808bfc0 msgr2=0x7fc95806c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:11.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.618+0000 7fc95f760640 1 -- 172.21.15.97:0/1162013493 shutdown_connections 2024-03-29T04:08:11.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.618+0000 7fc95f760640 1 -- 172.21.15.97:0/1162013493 wait complete. 2024-03-29T04:08:11.608 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-29T04:08:11.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:11.609 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-29T04:08:11.636 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T04:08:11.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:11.637 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T04:08:11.700 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:08:11.700 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:11.716 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:08:11.716 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:11.771 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:08:11.771 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:11.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.946+0000 7fa5a4160640 1 Processor -- start 2024-03-29T04:08:11.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.946+0000 7fa5a4160640 1 -- start start 2024-03-29T04:08:11.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.947+0000 7fa5a4160640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa59c15e930 0x7fa59c0cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.947+0000 7fa5a4160640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa59c0d3310 0x7fa59c06cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.947+0000 7fa5a4160640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa59c0cfb30 con 0x7fa59c0cf670 2024-03-29T04:08:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.947+0000 7fa5a4160640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa59c06c780 con 0x7fa59c15e930 2024-03-29T04:08:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.947+0000 7fa5a4160640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa59c0cfe10 con 0x7fa59c0d3310 2024-03-29T04:08:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.947+0000 7fa5a1efc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa59c15e930 0x7fa59c0cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.947+0000 7fa5a16fb640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa59c0cf670 0x7fa59c0d2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35384/0 (socket says 172.21.15.97:35384) 2024-03-29T04:08:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.947+0000 7fa5a16fb640 1 -- 172.21.15.97:0/2639762870 learned_addr learned my addr 172.21.15.97:0/2639762870 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.948+0000 7fa5a26fd640 1 --2- 172.21.15.97:0/2639762870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa59c0d3310 0x7fa59c06cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.948+0000 7fa5a1efc640 1 -- 172.21.15.97:0/2639762870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa59c0d3310 msgr2=0x7fa59c06cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.948+0000 7fa5a1efc640 1 --2- 172.21.15.97:0/2639762870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa59c0d3310 0x7fa59c06cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.948+0000 7fa5a1efc640 1 -- 172.21.15.97:0/2639762870 >> v1:172.21.15.97:6789/0 conn(0x7fa59c0cf670 legacy=0x7fa59c0d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.948+0000 7fa5a1efc640 1 -- 172.21.15.97:0/2639762870 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c0d00f0 con 0x7fa59c15e930 2024-03-29T04:08:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.948+0000 7fa5a0efa640 1 -- 172.21.15.97:0/2639762870 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2236255700 0 0) 0x7fa59c0cfb30 con 0x7fa59c0cf670 2024-03-29T04:08:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.948+0000 7fa5a1efc640 1 --2- 172.21.15.97:0/2639762870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa59c15e930 0x7fa59c0cf130 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa598009d90 tx=0x7fa59802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6dc37862c39655 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.949+0000 7fa5a0efa640 1 -- 172.21.15.97:0/2639762870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598039070 con 0x7fa59c15e930 2024-03-29T04:08:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.949+0000 7fa5a0efa640 1 -- 172.21.15.97:0/2639762870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59802cbc0 con 0x7fa59c15e930 2024-03-29T04:08:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.949+0000 7fa5a0efa640 1 -- 172.21.15.97:0/2639762870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598034440 con 0x7fa59c15e930 2024-03-29T04:08:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.949+0000 7fa5a4160640 1 -- 172.21.15.97:0/2639762870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa59c15e930 msgr2=0x7fa59c0cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.949+0000 7fa5a4160640 1 --2- 172.21.15.97:0/2639762870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa59c15e930 0x7fa59c0cf130 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa598009d90 tx=0x7fa59802be10 comp rx=0 tx=0).stop 2024-03-29T04:08:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.949+0000 7fa5a4160640 1 -- 172.21.15.97:0/2639762870 shutdown_connections 2024-03-29T04:08:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.949+0000 7fa5a4160640 1 --2- 172.21.15.97:0/2639762870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa59c0d3310 0x7fa59c06cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.949+0000 7fa5a4160640 1 --2- 172.21.15.97:0/2639762870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa59c15e930 0x7fa59c0cf130 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.949+0000 7fa5a4160640 1 -- 172.21.15.97:0/2639762870 >> 172.21.15.97:0/2639762870 conn(0x7fa59c08bfc0 msgr2=0x7fa59c0d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:11.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.949+0000 7fa5a4160640 1 -- 172.21.15.97:0/2639762870 shutdown_connections 2024-03-29T04:08:11.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.949+0000 7fa5a4160640 1 -- 172.21.15.97:0/2639762870 wait complete. 2024-03-29T04:08:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.950+0000 7fa5a4160640 1 Processor -- start 2024-03-29T04:08:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.950+0000 7fa5a4160640 1 -- start start 2024-03-29T04:08:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.951+0000 7fa5a4160640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa59c0cf670 0x7fa59c151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.951+0000 7fa5a4160640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa59c0d3310 0x7fa59c0d4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.951+0000 7fa5a4160640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa59c15e930 0x7fa59c0d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.951+0000 7fa5a4160640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c15ee90 con 0x7fa59c15e930 2024-03-29T04:08:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.951+0000 7fa5a4160640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c15ed10 con 0x7fa59c0d3310 2024-03-29T04:08:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.951+0000 7fa5a4160640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa59c0b3c50 con 0x7fa59c0cf670 2024-03-29T04:08:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.951+0000 7fa5a26fd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa59c15e930 0x7fa59c0d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.951+0000 7fa5a26fd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa59c15e930 0x7fa59c0d49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34616/0 (socket says 172.21.15.97:34616) 2024-03-29T04:08:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.951+0000 7fa5a26fd640 1 -- 172.21.15.97:0/1321987410 learned_addr learned my addr 172.21.15.97:0/1321987410 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.951+0000 7fa5a1efc640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa59c0cf670 0x7fa59c151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.951+0000 7fa5a16fb640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa59c0d3310 0x7fa59c0d4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.952+0000 7fa5a26fd640 1 -- 172.21.15.97:0/1321987410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa59c0cf670 msgr2=0x7fa59c151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.952+0000 7fa5a26fd640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa59c0cf670 0x7fa59c151d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.952+0000 7fa5a26fd640 1 -- 172.21.15.97:0/1321987410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa59c0d3310 msgr2=0x7fa59c0d4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.952+0000 7fa5a26fd640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa59c0d3310 0x7fa59c0d4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.952+0000 7fa5a26fd640 1 -- 172.21.15.97:0/1321987410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c1522b0 con 0x7fa59c15e930 2024-03-29T04:08:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.952+0000 7fa5a26fd640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa59c15e930 0x7fa59c0d49d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa594004650 tx=0x7fa59400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.952+0000 7fa5927fc640 1 -- 172.21.15.97:0/1321987410 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594017070 con 0x7fa59c15e930 2024-03-29T04:08:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.952+0000 7fa5927fc640 1 -- 172.21.15.97:0/1321987410 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59400c940 con 0x7fa59c15e930 2024-03-29T04:08:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.952+0000 7fa5a4160640 1 -- 172.21.15.97:0/1321987410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59c0cc9e0 con 0x7fa59c15e930 2024-03-29T04:08:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.952+0000 7fa5927fc640 1 -- 172.21.15.97:0/1321987410 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59400cd30 con 0x7fa59c15e930 2024-03-29T04:08:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.952+0000 7fa5a4160640 1 -- 172.21.15.97:0/1321987410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59c0cc700 con 0x7fa59c15e930 2024-03-29T04:08:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.953+0000 7fa5a4160640 1 -- 172.21.15.97:0/1321987410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59c068640 con 0x7fa59c15e930 2024-03-29T04:08:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.953+0000 7fa5927fc640 1 -- 172.21.15.97:0/1321987410 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa59401a070 con 0x7fa59c15e930 2024-03-29T04:08:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.954+0000 7fa5927fc640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa58006c530 0x7fa58006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.954+0000 7fa5927fc640 1 -- 172.21.15.97:0/1321987410 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa59408b510 con 0x7fa59c15e930 2024-03-29T04:08:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.954+0000 7fa5a1efc640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa58006c530 0x7fa58006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.954+0000 7fa5a1efc640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa58006c530 0x7fa58006e9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa598036000 tx=0x7fa598030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:11.958+0000 7fa5927fc640 1 -- 172.21.15.97:0/1321987410 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa59c1522b0 con 0x7fa59c15e930 2024-03-29T04:08:12.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.133+0000 7fa5a4160640 1 -- 172.21.15.97:0/1321987410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa59c06d020 con 0x7fa59c15e930 2024-03-29T04:08:12.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.134+0000 7fa5927fc640 1 -- 172.21.15.97:0/1321987410 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fa594059570 con 0x7fa59c15e930 2024-03-29T04:08:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:12.114 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.136+0000 7fa5a4160640 1 -- 172.21.15.97:0/1321987410 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa58006c530 msgr2=0x7fa58006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.136+0000 7fa5a4160640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa58006c530 0x7fa58006e9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa598036000 tx=0x7fa598030000 comp rx=0 tx=0).stop 2024-03-29T04:08:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.136+0000 7fa5a4160640 1 -- 172.21.15.97:0/1321987410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa59c15e930 msgr2=0x7fa59c0d49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.136+0000 7fa5a4160640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa59c15e930 0x7fa59c0d49d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa594004650 tx=0x7fa59400ece0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.136+0000 7fa5a4160640 1 -- 172.21.15.97:0/1321987410 shutdown_connections 2024-03-29T04:08:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.136+0000 7fa5a4160640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa58006c530 0x7fa58006e9f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.136+0000 7fa5a4160640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa59c15e930 0x7fa59c0d49d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.136+0000 7fa5a4160640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa59c0d3310 0x7fa59c0d4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.136+0000 7fa5a4160640 1 --2- 172.21.15.97:0/1321987410 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa59c0cf670 0x7fa59c151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.136+0000 7fa5a4160640 1 -- 172.21.15.97:0/1321987410 >> 172.21.15.97:0/1321987410 conn(0x7fa59c08bfc0 msgr2=0x7fa59c0cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:12.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.136+0000 7fa5a4160640 1 -- 172.21.15.97:0/1321987410 shutdown_connections 2024-03-29T04:08:12.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.136+0000 7fa5a4160640 1 -- 172.21.15.97:0/1321987410 wait complete. 2024-03-29T04:08:12.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:07:38.618704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:07:39.724339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:08:12.127 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:08:12.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.249+0000 7fee98959640 1 Processor -- start 2024-03-29T04:08:12.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.249+0000 7fee98959640 1 -- start start 2024-03-29T04:08:12.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.250+0000 7fee98959640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee940c6820 0x7fee940b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.250+0000 7fee98959640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fee940b2640 0x7fee940cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.250+0000 7fee98959640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee941604d0 con 0x7fee940b4e40 2024-03-29T04:08:12.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.250+0000 7fee98959640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fee940b2ab0 con 0x7fee940b2640 2024-03-29T04:08:12.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.250+0000 7fee98959640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fee940cba20 con 0x7fee940c6820 2024-03-29T04:08:12.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.250+0000 7fee92d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fee940b2640 0x7fee940cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.250+0000 7fee92d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fee940b2640 0x7fee940cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34618/0 (socket says 172.21.15.97:34618) 2024-03-29T04:08:12.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.250+0000 7fee91d9b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee940c6820 0x7fee940b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.250+0000 7fee92d9d640 1 -- 172.21.15.97:0/530195721 learned_addr learned my addr 172.21.15.97:0/530195721 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:12.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.251+0000 7fee92d9d640 1 -- 172.21.15.97:0/530195721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee940c6820 msgr2=0x7fee940b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:12.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.251+0000 7fee92d9d640 1 --2- 172.21.15.97:0/530195721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee940c6820 0x7fee940b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.251+0000 7fee9159a640 1 -- 172.21.15.97:0/530195721 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3809694467 0 0) 0x7fee941604d0 con 0x7fee940b4e40 2024-03-29T04:08:12.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.251+0000 7fee92d9d640 1 -- 172.21.15.97:0/530195721 >> v1:172.21.15.154:6789/0 conn(0x7fee940b4e40 legacy=0x7fee940b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.251+0000 7fee92d9d640 1 -- 172.21.15.97:0/530195721 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee941607b0 con 0x7fee940b2640 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.251+0000 7fee92d9d640 1 --2- 172.21.15.97:0/530195721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee940b2640 0x7fee940cbec0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0098a0 tx=0x7fee8c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcde7c1471e1524 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.252+0000 7fee9159a640 1 -- 172.21.15.97:0/530195721 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c035020 con 0x7fee940b2640 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.252+0000 7fee9159a640 1 -- 172.21.15.97:0/530195721 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8c031640 con 0x7fee940b2640 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.252+0000 7fee9159a640 1 -- 172.21.15.97:0/530195721 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c02d040 con 0x7fee940b2640 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.252+0000 7fee98959640 1 -- 172.21.15.97:0/530195721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee940b2640 msgr2=0x7fee940cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.252+0000 7fee98959640 1 --2- 172.21.15.97:0/530195721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee940b2640 0x7fee940cbec0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0098a0 tx=0x7fee8c02b8f0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.252+0000 7fee98959640 1 -- 172.21.15.97:0/530195721 shutdown_connections 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.252+0000 7fee98959640 1 --2- 172.21.15.97:0/530195721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee940b2640 0x7fee940cbec0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.252+0000 7fee98959640 1 --2- 172.21.15.97:0/530195721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee940c6820 0x7fee940b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.252+0000 7fee98959640 1 -- 172.21.15.97:0/530195721 >> 172.21.15.97:0/530195721 conn(0x7fee940d0450 msgr2=0x7fee940d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.252+0000 7fee98959640 1 -- 172.21.15.97:0/530195721 shutdown_connections 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.252+0000 7fee98959640 1 -- 172.21.15.97:0/530195721 wait complete. 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.254+0000 7fee98959640 1 Processor -- start 2024-03-29T04:08:12.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.254+0000 7fee98959640 1 -- start start 2024-03-29T04:08:12.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.254+0000 7fee98959640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee940b2640 0x7fee940ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.254+0000 7fee98959640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee940b4e40 0x7fee94156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.254+0000 7fee98959640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee940c6820 0x7fee94157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.254+0000 7fee98959640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fee940c6c80 con 0x7fee940b2640 2024-03-29T04:08:12.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.254+0000 7fee98959640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fee94160d70 con 0x7fee940b4e40 2024-03-29T04:08:12.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.254+0000 7fee98959640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fee940c60f0 con 0x7fee940c6820 2024-03-29T04:08:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.254+0000 7fee9259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee940b2640 0x7fee940ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.254+0000 7fee9259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee940b2640 0x7fee940ad500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34622/0 (socket says 172.21.15.97:34622) 2024-03-29T04:08:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.254+0000 7fee9259c640 1 -- 172.21.15.97:0/651537291 learned_addr learned my addr 172.21.15.97:0/651537291 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.254+0000 7fee91d9b640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee940b4e40 0x7fee94156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.254+0000 7fee92d9d640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee940c6820 0x7fee94157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.255+0000 7fee9259c640 1 -- 172.21.15.97:0/651537291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee940c6820 msgr2=0x7fee94157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.255+0000 7fee9259c640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee940c6820 0x7fee94157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.255+0000 7fee9259c640 1 -- 172.21.15.97:0/651537291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee940b4e40 msgr2=0x7fee94156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.255+0000 7fee9259c640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee940b4e40 0x7fee94156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.255+0000 7fee9259c640 1 -- 172.21.15.97:0/651537291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee940ae870 con 0x7fee940b2640 2024-03-29T04:08:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.255+0000 7fee9259c640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee940b2640 0x7fee940ad500 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fee8400c8c0 tx=0x7fee8400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.255+0000 7fee82ffd640 1 -- 172.21.15.97:0/651537291 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84017070 con 0x7fee940b2640 2024-03-29T04:08:12.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.255+0000 7fee82ffd640 1 -- 172.21.15.97:0/651537291 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee84003dc0 con 0x7fee940b2640 2024-03-29T04:08:12.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.255+0000 7fee82ffd640 1 -- 172.21.15.97:0/651537291 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840052b0 con 0x7fee940b2640 2024-03-29T04:08:12.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.255+0000 7fee98959640 1 -- 172.21.15.97:0/651537291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee940aeaf0 con 0x7fee940b2640 2024-03-29T04:08:12.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.255+0000 7fee98959640 1 -- 172.21.15.97:0/651537291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee940af0b0 con 0x7fee940b2640 2024-03-29T04:08:12.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.256+0000 7fee98959640 1 -- 172.21.15.97:0/651537291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee940723f0 con 0x7fee940b2640 2024-03-29T04:08:12.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.257+0000 7fee82ffd640 1 -- 172.21.15.97:0/651537291 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fee8401a070 con 0x7fee940b2640 2024-03-29T04:08:12.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.257+0000 7fee82ffd640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fee7406c530 0x7fee7406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.257+0000 7fee82ffd640 1 -- 172.21.15.97:0/651537291 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fee8408acf0 con 0x7fee940b2640 2024-03-29T04:08:12.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.257+0000 7fee91d9b640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fee7406c530 0x7fee7406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.257+0000 7fee91d9b640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fee7406c530 0x7fee7406e9f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fee940cc450 tx=0x7fee88009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:12.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.261+0000 7fee82ffd640 1 -- 172.21.15.97:0/651537291 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fee8408f070 con 0x7fee940b2640 2024-03-29T04:08:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.435+0000 7fee98959640 1 -- 172.21.15.97:0/651537291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fee9404f9d0 con 0x7fee940b2640 2024-03-29T04:08:12.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.436+0000 7fee82ffd640 1 -- 172.21.15.97:0/651537291 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fee940ae870 con 0x7fee940b2640 2024-03-29T04:08:12.416 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:08:12.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.438+0000 7fee98959640 1 -- 172.21.15.97:0/651537291 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fee7406c530 msgr2=0x7fee7406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.438+0000 7fee98959640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fee7406c530 0x7fee7406e9f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fee940cc450 tx=0x7fee88009190 comp rx=0 tx=0).stop 2024-03-29T04:08:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.438+0000 7fee98959640 1 -- 172.21.15.97:0/651537291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee940b2640 msgr2=0x7fee940ad500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.438+0000 7fee98959640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee940b2640 0x7fee940ad500 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fee8400c8c0 tx=0x7fee8400cd80 comp rx=0 tx=0).stop 2024-03-29T04:08:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.438+0000 7fee98959640 1 -- 172.21.15.97:0/651537291 shutdown_connections 2024-03-29T04:08:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.438+0000 7fee98959640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fee7406c530 0x7fee7406e9f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.438+0000 7fee98959640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee940c6820 0x7fee94157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.438+0000 7fee98959640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee940b4e40 0x7fee94156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.438+0000 7fee98959640 1 --2- 172.21.15.97:0/651537291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee940b2640 0x7fee940ad500 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.438+0000 7fee98959640 1 -- 172.21.15.97:0/651537291 >> 172.21.15.97:0/651537291 conn(0x7fee940d0450 msgr2=0x7fee94158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.438+0000 7fee98959640 1 -- 172.21.15.97:0/651537291 shutdown_connections 2024-03-29T04:08:12.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.438+0000 7fee98959640 1 -- 172.21.15.97:0/651537291 wait complete. 2024-03-29T04:08:12.428 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:08:12.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.552+0000 7fc482550640 1 Processor -- start 2024-03-29T04:08:12.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.552+0000 7fc482550640 1 -- start start 2024-03-29T04:08:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.553+0000 7fc482550640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc47c06bc20 0x7fc47c15eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.553+0000 7fc482550640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc47c0b1c10 con 0x7fc47c0cef50 2024-03-29T04:08:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.553+0000 7fc482550640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc47c0b0e50 con 0x7fc47c15c480 2024-03-29T04:08:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.553+0000 7fc482550640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fc47c0b2d20 con 0x7fc47c06bc20 2024-03-29T04:08:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.553+0000 7fc47b7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc47c0cef50 0x7fc47c06b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35394/0 (socket says 172.21.15.97:35394) 2024-03-29T04:08:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.553+0000 7fc47b7fe640 1 -- 172.21.15.97:0/1340632153 learned_addr learned my addr 172.21.15.97:0/1340632153 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.554+0000 7fc480aed640 1 --2- 172.21.15.97:0/1340632153 >> v2:172.21.15.154:3300/0 conn(0x7fc47c06bc20 0x7fc47c15eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.554+0000 7fc47affd640 1 -- 172.21.15.97:0/1340632153 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1124027869 0 0) 0x7fc47c0b1c10 con 0x7fc47c0cef50 2024-03-29T04:08:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.554+0000 7fc47affd640 1 -- 172.21.15.97:0/1340632153 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc464003520 con 0x7fc47c0cef50 2024-03-29T04:08:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.554+0000 7fc47affd640 1 -- 172.21.15.97:0/1340632153 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1859289207 0 0) 0x7fc47c0b0e50 con 0x7fc47c15c480 2024-03-29T04:08:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.554+0000 7fc47affd640 1 -- 172.21.15.97:0/1340632153 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc47c0b1c10 con 0x7fc47c15c480 2024-03-29T04:08:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.554+0000 7fc480aed640 1 -- 172.21.15.97:0/1340632153 >> v1:172.21.15.154:6789/0 conn(0x7fc47c15c480 legacy=0x7fc47c0b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.554+0000 7fc480aed640 1 -- 172.21.15.97:0/1340632153 >> v1:172.21.15.97:6789/0 conn(0x7fc47c0cef50 legacy=0x7fc47c06b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.554+0000 7fc480aed640 1 -- 172.21.15.97:0/1340632153 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47c0b1130 con 0x7fc47c06bc20 2024-03-29T04:08:12.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.555+0000 7fc480aed640 1 --2- 172.21.15.97:0/1340632153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc47c06bc20 0x7fc47c15eeb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc474009d90 tx=0x7fc47402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4963f96bdf89af19 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:12.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.555+0000 7fc47affd640 1 -- 172.21.15.97:0/1340632153 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474039070 con 0x7fc47c06bc20 2024-03-29T04:08:12.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.555+0000 7fc47affd640 1 -- 172.21.15.97:0/1340632153 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc47402cc60 con 0x7fc47c06bc20 2024-03-29T04:08:12.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.555+0000 7fc47affd640 1 -- 172.21.15.97:0/1340632153 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474034640 con 0x7fc47c06bc20 2024-03-29T04:08:12.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.555+0000 7fc482550640 1 -- 172.21.15.97:0/1340632153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc47c06bc20 msgr2=0x7fc47c15eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.555+0000 7fc482550640 1 --2- 172.21.15.97:0/1340632153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc47c06bc20 0x7fc47c15eeb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc474009d90 tx=0x7fc47402beb0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.555+0000 7fc482550640 1 -- 172.21.15.97:0/1340632153 shutdown_connections 2024-03-29T04:08:12.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.555+0000 7fc482550640 1 --2- 172.21.15.97:0/1340632153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc47c06bc20 0x7fc47c15eeb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.555+0000 7fc482550640 1 -- 172.21.15.97:0/1340632153 >> 172.21.15.97:0/1340632153 conn(0x7fc47c0b2ed0 msgr2=0x7fc47c0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:12.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.555+0000 7fc482550640 1 -- 172.21.15.97:0/1340632153 shutdown_connections 2024-03-29T04:08:12.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.555+0000 7fc482550640 1 -- 172.21.15.97:0/1340632153 wait complete. 2024-03-29T04:08:12.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.557+0000 7fc482550640 1 Processor -- start 2024-03-29T04:08:12.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.557+0000 7fc482550640 1 -- start start 2024-03-29T04:08:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.557+0000 7fc482550640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc47c06bc20 0x7fc47c1513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.557+0000 7fc482550640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47c0cef50 0x7fc47c1e5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.557+0000 7fc482550640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc47c15c480 0x7fc47c1e5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.557+0000 7fc482550640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c15c860 con 0x7fc47c0cef50 2024-03-29T04:08:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.557+0000 7fc482550640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c0b1780 con 0x7fc47c06bc20 2024-03-29T04:08:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.557+0000 7fc482550640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fc47c15f3f0 con 0x7fc47c15c480 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc47b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47c0cef50 0x7fc47c1e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc47b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47c0cef50 0x7fc47c1e5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34624/0 (socket says 172.21.15.97:34624) 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc47bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc47c06bc20 0x7fc47c1513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc47b7fe640 1 -- 172.21.15.97:0/61409470 learned_addr learned my addr 172.21.15.97:0/61409470 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc480aed640 1 --2- 172.21.15.97:0/61409470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc47c15c480 0x7fc47c1e5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc47b7fe640 1 -- 172.21.15.97:0/61409470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc47c15c480 msgr2=0x7fc47c1e5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc47b7fe640 1 --2- 172.21.15.97:0/61409470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc47c15c480 0x7fc47c1e5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc47b7fe640 1 -- 172.21.15.97:0/61409470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc47c06bc20 msgr2=0x7fc47c1513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc47b7fe640 1 --2- 172.21.15.97:0/61409470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc47c06bc20 0x7fc47c1513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc47b7fe640 1 -- 172.21.15.97:0/61409470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47c0b4040 con 0x7fc47c0cef50 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc47b7fe640 1 --2- 172.21.15.97:0/61409470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47c0cef50 0x7fc47c1e5450 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc47000c500 tx=0x7fc47000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc478ff9640 1 -- 172.21.15.97:0/61409470 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470017070 con 0x7fc47c0cef50 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc478ff9640 1 -- 172.21.15.97:0/61409470 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc470005d00 con 0x7fc47c0cef50 2024-03-29T04:08:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc482550640 1 -- 172.21.15.97:0/61409470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47c0b4320 con 0x7fc47c0cef50 2024-03-29T04:08:12.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc478ff9640 1 -- 172.21.15.97:0/61409470 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470007330 con 0x7fc47c0cef50 2024-03-29T04:08:12.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.558+0000 7fc482550640 1 -- 172.21.15.97:0/61409470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47c0b48e0 con 0x7fc47c0cef50 2024-03-29T04:08:12.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.559+0000 7fc482550640 1 -- 172.21.15.97:0/61409470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc47c09f530 con 0x7fc47c0cef50 2024-03-29T04:08:12.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.560+0000 7fc478ff9640 1 -- 172.21.15.97:0/61409470 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc47001a070 con 0x7fc47c0cef50 2024-03-29T04:08:12.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.560+0000 7fc478ff9640 1 --2- 172.21.15.97:0/61409470 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc46806c560 0x7fc46806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.560+0000 7fc478ff9640 1 -- 172.21.15.97:0/61409470 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc47008aa20 con 0x7fc47c0cef50 2024-03-29T04:08:12.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.560+0000 7fc47bfff640 1 --2- 172.21.15.97:0/61409470 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc46806c560 0x7fc46806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.560+0000 7fc47bfff640 1 --2- 172.21.15.97:0/61409470 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc46806c560 0x7fc46806ea20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc46c007fd0 tx=0x7fc46c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:12.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.564+0000 7fc478ff9640 1 -- 172.21.15.97:0/61409470 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc470058b40 con 0x7fc47c0cef50 2024-03-29T04:08:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.740+0000 7fc482550640 1 -- 172.21.15.97:0/61409470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc47c06b510 con 0x7fc47c0cef50 2024-03-29T04:08:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.740+0000 7fc478ff9640 1 -- 172.21.15.97:0/61409470 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc47005ce40 con 0x7fc47c0cef50 2024-03-29T04:08:12.720 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:08:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.743+0000 7fc482550640 1 -- 172.21.15.97:0/61409470 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc46806c560 msgr2=0x7fc46806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.743+0000 7fc482550640 1 --2- 172.21.15.97:0/61409470 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc46806c560 0x7fc46806ea20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc46c007fd0 tx=0x7fc46c009040 comp rx=0 tx=0).stop 2024-03-29T04:08:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.743+0000 7fc482550640 1 -- 172.21.15.97:0/61409470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47c0cef50 msgr2=0x7fc47c1e5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.743+0000 7fc482550640 1 --2- 172.21.15.97:0/61409470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47c0cef50 0x7fc47c1e5450 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc47000c500 tx=0x7fc47000c9c0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.743+0000 7fc482550640 1 -- 172.21.15.97:0/61409470 shutdown_connections 2024-03-29T04:08:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.743+0000 7fc482550640 1 --2- 172.21.15.97:0/61409470 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fc46806c560 0x7fc46806ea20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.743+0000 7fc482550640 1 --2- 172.21.15.97:0/61409470 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc47c15c480 0x7fc47c1e5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.743+0000 7fc482550640 1 --2- 172.21.15.97:0/61409470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc47c0cef50 0x7fc47c1e5450 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.743+0000 7fc482550640 1 --2- 172.21.15.97:0/61409470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc47c06bc20 0x7fc47c1513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.743+0000 7fc482550640 1 -- 172.21.15.97:0/61409470 >> 172.21.15.97:0/61409470 conn(0x7fc47c0b2ed0 msgr2=0x7fc47c15bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:12.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.743+0000 7fc482550640 1 -- 172.21.15.97:0/61409470 shutdown_connections 2024-03-29T04:08:12.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.743+0000 7fc482550640 1 -- 172.21.15.97:0/61409470 wait complete. 2024-03-29T04:08:12.733 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:08:12.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.857+0000 7f943271c640 1 Processor -- start 2024-03-29T04:08:12.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.857+0000 7f943271c640 1 -- start start 2024-03-29T04:08:12.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.857+0000 7f943271c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f942c15e920 0x7f942c0d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.857+0000 7f943271c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f942c0cfde0 0x7f942c0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.857+0000 7f943271c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f942c0d3780 con 0x7f942c0d0f90 2024-03-29T04:08:12.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.857+0000 7f943271c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f942c15ed00 con 0x7f942c0cfde0 2024-03-29T04:08:12.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.857+0000 7f943271c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f942c0d21e0 con 0x7f942c15e920 2024-03-29T04:08:12.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.858+0000 7f942b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f942c0cfde0 0x7f942c0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.858+0000 7f9430cb9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f942c0d0f90 0x7f942c06c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35398/0 (socket says 172.21.15.97:35398) 2024-03-29T04:08:12.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.858+0000 7f9430cb9640 1 -- 172.21.15.97:0/648550027 learned_addr learned my addr 172.21.15.97:0/648550027 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:12.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.858+0000 7f942bfff640 1 --2- 172.21.15.97:0/648550027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f942c15e920 0x7f942c0d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.858+0000 7f942b7fe640 1 -- 172.21.15.97:0/648550027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f942c15e920 msgr2=0x7f942c0d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:12.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.858+0000 7f942b7fe640 1 --2- 172.21.15.97:0/648550027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f942c15e920 0x7f942c0d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.858+0000 7f942b7fe640 1 -- 172.21.15.97:0/648550027 >> v1:172.21.15.97:6789/0 conn(0x7f942c0d0f90 legacy=0x7f942c06c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.858+0000 7f942affd640 1 -- 172.21.15.97:0/648550027 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 152149153 0 0) 0x7f9424001490 con 0x7f942c0d0f90 2024-03-29T04:08:12.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.858+0000 7f942b7fe640 1 -- 172.21.15.97:0/648550027 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c0d2480 con 0x7f942c0cfde0 2024-03-29T04:08:12.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.859+0000 7f942b7fe640 1 --2- 172.21.15.97:0/648550027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0cfde0 0x7f942c0d0240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f94200097d0 tx=0x7f942002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0df76935a59603 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:12.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.859+0000 7f942affd640 1 -- 172.21.15.97:0/648550027 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420035020 con 0x7f942c0cfde0 2024-03-29T04:08:12.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.859+0000 7f942affd640 1 -- 172.21.15.97:0/648550027 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9420031800 con 0x7f942c0cfde0 2024-03-29T04:08:12.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.859+0000 7f942affd640 1 -- 172.21.15.97:0/648550027 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420031bd0 con 0x7f942c0cfde0 2024-03-29T04:08:12.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.859+0000 7f943271c640 1 -- 172.21.15.97:0/648550027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0cfde0 msgr2=0x7f942c0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.859+0000 7f943271c640 1 --2- 172.21.15.97:0/648550027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0cfde0 0x7f942c0d0240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f94200097d0 tx=0x7f942002b8a0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.859+0000 7f943271c640 1 -- 172.21.15.97:0/648550027 shutdown_connections 2024-03-29T04:08:12.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.859+0000 7f943271c640 1 --2- 172.21.15.97:0/648550027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0cfde0 0x7f942c0d0240 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.859+0000 7f943271c640 1 --2- 172.21.15.97:0/648550027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f942c15e920 0x7f942c0d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.859+0000 7f943271c640 1 -- 172.21.15.97:0/648550027 >> 172.21.15.97:0/648550027 conn(0x7f942c08bfc0 msgr2=0x7f942c0d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.859+0000 7f943271c640 1 -- 172.21.15.97:0/648550027 shutdown_connections 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.859+0000 7f943271c640 1 -- 172.21.15.97:0/648550027 wait complete. 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.861+0000 7f943271c640 1 Processor -- start 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.861+0000 7f943271c640 1 -- start start 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.861+0000 7f943271c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f942c0cfde0 0x7f942c0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.861+0000 7f943271c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0d0f90 0x7f942c0c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.861+0000 7f943271c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f942c15e920 0x7f942c0b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.861+0000 7f943271c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f942c0d2d20 con 0x7f942c0d0f90 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.861+0000 7f943271c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f942c0d2a40 con 0x7f942c15e920 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.861+0000 7f943271c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f942c0d4ef0 con 0x7f942c0cfde0 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f942b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0d0f90 0x7f942c0c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f942b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0d0f90 0x7f942c0c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34636/0 (socket says 172.21.15.97:34636) 2024-03-29T04:08:12.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f942b7fe640 1 -- 172.21.15.97:0/2766094906 learned_addr learned my addr 172.21.15.97:0/2766094906 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:12.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f9430cb9640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f942c15e920 0x7f942c0b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f942bfff640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f942c0cfde0 0x7f942c0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f942b7fe640 1 -- 172.21.15.97:0/2766094906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f942c0cfde0 msgr2=0x7f942c0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f942b7fe640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f942c0cfde0 0x7f942c0cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f942b7fe640 1 -- 172.21.15.97:0/2766094906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f942c15e920 msgr2=0x7f942c0b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:12.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f942b7fe640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f942c15e920 0x7f942c0b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:12.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f942b7fe640 1 -- 172.21.15.97:0/2766094906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c14b7d0 con 0x7f942c0d0f90 2024-03-29T04:08:12.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f942b7fe640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0d0f90 0x7f942c0c9b30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9420034040 tx=0x7f94200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:12.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f9428ff9640 1 -- 172.21.15.97:0/2766094906 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420040070 con 0x7f942c0d0f90 2024-03-29T04:08:12.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f9428ff9640 1 -- 172.21.15.97:0/2766094906 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9420039cf0 con 0x7f942c0d0f90 2024-03-29T04:08:12.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f943271c640 1 -- 172.21.15.97:0/2766094906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942c14bab0 con 0x7f942c0d0f90 2024-03-29T04:08:12.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f9428ff9640 1 -- 172.21.15.97:0/2766094906 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420035040 con 0x7f942c0d0f90 2024-03-29T04:08:12.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.862+0000 7f943271c640 1 -- 172.21.15.97:0/2766094906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942c14c070 con 0x7f942c0d0f90 2024-03-29T04:08:12.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.863+0000 7f943271c640 1 -- 172.21.15.97:0/2766094906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942c0a2560 con 0x7f942c0d0f90 2024-03-29T04:08:12.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.864+0000 7f9428ff9640 1 -- 172.21.15.97:0/2766094906 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9420038020 con 0x7f942c0d0f90 2024-03-29T04:08:12.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.864+0000 7f9428ff9640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f941806c530 0x7f941806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:12.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.864+0000 7f9428ff9640 1 -- 172.21.15.97:0/2766094906 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f94200ad0b0 con 0x7f942c0d0f90 2024-03-29T04:08:12.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.864+0000 7f942bfff640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f941806c530 0x7f941806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:12.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.865+0000 7f942bfff640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f941806c530 0x7f941806e9f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f941c009b40 tx=0x7f941c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:12.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:12.868+0000 7f9428ff9640 1 -- 172.21.15.97:0/2766094906 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f942007b080 con 0x7f942c0d0f90 2024-03-29T04:08:13.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.042+0000 7f943271c640 1 -- 172.21.15.97:0/2766094906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f942c04f9d0 con 0x7f942c0d0f90 2024-03-29T04:08:13.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.042+0000 7f9428ff9640 1 -- 172.21.15.97:0/2766094906 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f942007f380 con 0x7f942c0d0f90 2024-03-29T04:08:13.022 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:08:13.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.044+0000 7f943271c640 1 -- 172.21.15.97:0/2766094906 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f941806c530 msgr2=0x7f941806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.044+0000 7f943271c640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f941806c530 0x7f941806e9f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f941c009b40 tx=0x7f941c009190 comp rx=0 tx=0).stop 2024-03-29T04:08:13.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.044+0000 7f943271c640 1 -- 172.21.15.97:0/2766094906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0d0f90 msgr2=0x7f942c0c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.044+0000 7f943271c640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0d0f90 0x7f942c0c9b30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9420034040 tx=0x7f94200046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.044+0000 7f943271c640 1 -- 172.21.15.97:0/2766094906 shutdown_connections 2024-03-29T04:08:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.044+0000 7f943271c640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f941806c530 0x7f941806e9f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.044+0000 7f943271c640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f942c15e920 0x7f942c0b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.044+0000 7f943271c640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0d0f90 0x7f942c0c9b30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.044+0000 7f943271c640 1 --2- 172.21.15.97:0/2766094906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f942c0cfde0 0x7f942c0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.044+0000 7f943271c640 1 -- 172.21.15.97:0/2766094906 >> 172.21.15.97:0/2766094906 conn(0x7f942c08bfc0 msgr2=0x7f942c160e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.044+0000 7f943271c640 1 -- 172.21.15.97:0/2766094906 shutdown_connections 2024-03-29T04:08:13.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.044+0000 7f943271c640 1 -- 172.21.15.97:0/2766094906 wait complete. 2024-03-29T04:08:13.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:08:13.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.157+0000 7ff05d39d640 1 Processor -- start 2024-03-29T04:08:13.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.157+0000 7ff05d39d640 1 -- start start 2024-03-29T04:08:13.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff05d39d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff05815e780 0x7ff0580cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff05d39d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff0580ba000 0x7ff0580d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff05d39d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0580cfe70 con 0x7ff05815eb60 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff05d39d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ff0580b7f50 con 0x7ff0580ba000 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff05d39d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff0580b8230 con 0x7ff05815e780 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff0567fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff05815eb60 0x7ff0580cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35406/0 (socket says 172.21.15.97:35406) 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff0567fc640 1 -- 172.21.15.97:0/1124418924 learned_addr learned my addr 172.21.15.97:0/1124418924 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff0577fe640 1 --2- 172.21.15.97:0/1124418924 >> v2:172.21.15.154:3300/0 conn(0x7ff0580ba000 0x7ff0580d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff056ffd640 1 --2- 172.21.15.97:0/1124418924 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff05815e780 0x7ff0580cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff055ffb640 1 -- 172.21.15.97:0/1124418924 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3558506303 0 0) 0x7ff0580cfe70 con 0x7ff05815eb60 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff055ffb640 1 -- 172.21.15.97:0/1124418924 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff040003520 con 0x7ff05815eb60 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff0577fe640 1 -- 172.21.15.97:0/1124418924 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff05815e780 msgr2=0x7ff0580cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff0577fe640 1 --2- 172.21.15.97:0/1124418924 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff05815e780 0x7ff0580cebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff0577fe640 1 -- 172.21.15.97:0/1124418924 >> v1:172.21.15.97:6789/0 conn(0x7ff05815eb60 legacy=0x7ff0580cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff0577fe640 1 -- 172.21.15.97:0/1124418924 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0580b8510 con 0x7ff0580ba000 2024-03-29T04:08:13.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.158+0000 7ff056ffd640 1 --2- 172.21.15.97:0/1124418924 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff05815e780 0x7ff0580cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:13.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.159+0000 7ff0577fe640 1 --2- 172.21.15.97:0/1124418924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0580ba000 0x7ff0580d2570 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff050009d90 tx=0x7ff05002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0418f675f58e0f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:13.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.159+0000 7ff055ffb640 1 -- 172.21.15.97:0/1124418924 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050039070 con 0x7ff0580ba000 2024-03-29T04:08:13.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.159+0000 7ff055ffb640 1 -- 172.21.15.97:0/1124418924 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff05002cc50 con 0x7ff0580ba000 2024-03-29T04:08:13.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.159+0000 7ff055ffb640 1 -- 172.21.15.97:0/1124418924 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050034440 con 0x7ff0580ba000 2024-03-29T04:08:13.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.160+0000 7ff05d39d640 1 -- 172.21.15.97:0/1124418924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0580ba000 msgr2=0x7ff0580d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.160+0000 7ff05d39d640 1 --2- 172.21.15.97:0/1124418924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0580ba000 0x7ff0580d2570 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff050009d90 tx=0x7ff05002bea0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.160+0000 7ff05d39d640 1 -- 172.21.15.97:0/1124418924 shutdown_connections 2024-03-29T04:08:13.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.160+0000 7ff05d39d640 1 --2- 172.21.15.97:0/1124418924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0580ba000 0x7ff0580d2570 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.160+0000 7ff05d39d640 1 --2- 172.21.15.97:0/1124418924 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff05815e780 0x7ff0580cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.160+0000 7ff05d39d640 1 -- 172.21.15.97:0/1124418924 >> 172.21.15.97:0/1124418924 conn(0x7ff05808bfc0 msgr2=0x7ff0580d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:13.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.160+0000 7ff05d39d640 1 -- 172.21.15.97:0/1124418924 shutdown_connections 2024-03-29T04:08:13.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.160+0000 7ff05d39d640 1 -- 172.21.15.97:0/1124418924 wait complete. 2024-03-29T04:08:13.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.161+0000 7ff05d39d640 1 Processor -- start 2024-03-29T04:08:13.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.161+0000 7ff05d39d640 1 -- start start 2024-03-29T04:08:13.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff05d39d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0580ba000 0x7ff05809f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:13.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff05d39d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05815e780 0x7ff0580cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:13.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff05d39d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff05815b690 0x7ff0580cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:13.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff05d39d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff0580cf810 con 0x7ff05815e780 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff05d39d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff0580b8ad0 con 0x7ff05815b690 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff05d39d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff0580cf990 con 0x7ff0580ba000 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff0567fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05815e780 0x7ff0580cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff0567fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05815e780 0x7ff0580cc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34652/0 (socket says 172.21.15.97:34652) 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff0567fc640 1 -- 172.21.15.97:0/1707591728 learned_addr learned my addr 172.21.15.97:0/1707591728 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff0577fe640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff05815b690 0x7ff0580cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff056ffd640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0580ba000 0x7ff05809f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff0567fc640 1 -- 172.21.15.97:0/1707591728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0580ba000 msgr2=0x7ff05809f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff0567fc640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0580ba000 0x7ff05809f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff0567fc640 1 -- 172.21.15.97:0/1707591728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff05815b690 msgr2=0x7ff0580cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff0567fc640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff05815b690 0x7ff0580cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff0567fc640 1 -- 172.21.15.97:0/1707591728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05809e550 con 0x7ff05815e780 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff0567fc640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05815e780 0x7ff0580cc9f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00c390 tx=0x7ff04c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:13.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff03f7fe640 1 -- 172.21.15.97:0/1707591728 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c017070 con 0x7ff05815e780 2024-03-29T04:08:13.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff03f7fe640 1 -- 172.21.15.97:0/1707591728 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff04c005ce0 con 0x7ff05815e780 2024-03-29T04:08:13.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff05d39d640 1 -- 172.21.15.97:0/1707591728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05809e830 con 0x7ff05815e780 2024-03-29T04:08:13.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff03f7fe640 1 -- 172.21.15.97:0/1707591728 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c0072f0 con 0x7ff05815e780 2024-03-29T04:08:13.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.162+0000 7ff05d39d640 1 -- 172.21.15.97:0/1707591728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05809edf0 con 0x7ff05815e780 2024-03-29T04:08:13.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.163+0000 7ff05d39d640 1 -- 172.21.15.97:0/1707591728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff058068640 con 0x7ff05815e780 2024-03-29T04:08:13.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.164+0000 7ff03f7fe640 1 -- 172.21.15.97:0/1707591728 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff04c01a070 con 0x7ff05815e780 2024-03-29T04:08:13.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.164+0000 7ff03f7fe640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff04406c530 0x7ff04406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:13.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.164+0000 7ff03f7fe640 1 -- 172.21.15.97:0/1707591728 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff04c08a9b0 con 0x7ff05815e780 2024-03-29T04:08:13.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.164+0000 7ff056ffd640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff04406c530 0x7ff04406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:13.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.165+0000 7ff056ffd640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff04406c530 0x7ff04406e9f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff048009560 tx=0x7ff048009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:13.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.168+0000 7ff03f7fe640 1 -- 172.21.15.97:0/1707591728 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff04c058a10 con 0x7ff05815e780 2024-03-29T04:08:13.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.354+0000 7ff05d39d640 1 -- 172.21.15.97:0/1707591728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff0581e73d0 con 0x7ff05815e780 2024-03-29T04:08:13.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.354+0000 7ff03f7fe640 1 -- 172.21.15.97:0/1707591728 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7ff04c05cd10 con 0x7ff05815e780 2024-03-29T04:08:13.334 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-03-29T04:08:13.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.356+0000 7ff05d39d640 1 -- 172.21.15.97:0/1707591728 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff04406c530 msgr2=0x7ff04406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.356+0000 7ff05d39d640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff04406c530 0x7ff04406e9f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff048009560 tx=0x7ff048009190 comp rx=0 tx=0).stop 2024-03-29T04:08:13.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.356+0000 7ff05d39d640 1 -- 172.21.15.97:0/1707591728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05815e780 msgr2=0x7ff0580cc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.356+0000 7ff05d39d640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05815e780 0x7ff0580cc9f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00c390 tx=0x7ff04c00c850 comp rx=0 tx=0).stop 2024-03-29T04:08:13.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.356+0000 7ff05d39d640 1 -- 172.21.15.97:0/1707591728 shutdown_connections 2024-03-29T04:08:13.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.356+0000 7ff05d39d640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff04406c530 0x7ff04406e9f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.356+0000 7ff05d39d640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff05815b690 0x7ff0580cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.356+0000 7ff05d39d640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05815e780 0x7ff0580cc9f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.356+0000 7ff05d39d640 1 --2- 172.21.15.97:0/1707591728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0580ba000 0x7ff05809f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.356+0000 7ff05d39d640 1 -- 172.21.15.97:0/1707591728 >> 172.21.15.97:0/1707591728 conn(0x7ff05808bfc0 msgr2=0x7ff05806c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:13.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.356+0000 7ff05d39d640 1 -- 172.21.15.97:0/1707591728 shutdown_connections 2024-03-29T04:08:13.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.356+0000 7ff05d39d640 1 -- 172.21.15.97:0/1707591728 wait complete. 2024-03-29T04:08:13.347 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-29T04:08:13.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:13.347 DEBUG:teuthology.orchestra.run.smithi154:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-29T04:08:13.375 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T04:08:13.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:13.376 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T04:08:13.463 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:08:13.463 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:13.478 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:08:13.478 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:13.534 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:08:13.534 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:13.589 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:13.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.747+0000 7fdfd41d5640 1 Processor -- start 2024-03-29T04:08:13.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.748+0000 7fdfd41d5640 1 -- start start 2024-03-29T04:08:13.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.748+0000 7fdfd41d5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdfcc06f3f0 0x7fdfcc072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:13.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.748+0000 7fdfd41d5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfcc071ae0 con 0x7fdfcc0730c0 2024-03-29T04:08:13.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.748+0000 7fdfd41d5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfcc071dc0 con 0x7fdfcc15b370 2024-03-29T04:08:13.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.748+0000 7fdfd41d5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fdfcc0b39d0 con 0x7fdfcc06f3f0 2024-03-29T04:08:13.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.749+0000 7fdfd1770640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdfcc06f3f0 0x7fdfcc072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:13.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.749+0000 7fdfd2772640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdfcc0730c0 0x7fdfcc15f0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35408/0 (socket says 172.21.15.97:35408) 2024-03-29T04:08:13.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.749+0000 7fdfd2772640 1 -- 172.21.15.97:0/996790477 learned_addr learned my addr 172.21.15.97:0/996790477 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:13.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.749+0000 7fdfd1770640 1 -- 172.21.15.97:0/996790477 >> v1:172.21.15.154:6789/0 conn(0x7fdfcc15b370 legacy=0x7fdfcc0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.749+0000 7fdfd1770640 1 -- 172.21.15.97:0/996790477 >> v1:172.21.15.97:6789/0 conn(0x7fdfcc0730c0 legacy=0x7fdfcc15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.749+0000 7fdfd1770640 1 -- 172.21.15.97:0/996790477 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc0720a0 con 0x7fdfcc06f3f0 2024-03-29T04:08:13.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.749+0000 7fdfd0f6f640 1 -- 172.21.15.97:0/996790477 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 502955483 0 0) 0x7fdfcc071ae0 con 0x7fdfcc0730c0 2024-03-29T04:08:13.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.750+0000 7fdfd1770640 1 --2- 172.21.15.97:0/996790477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc06f3f0 0x7fdfcc072b80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0097a0 tx=0x7fdfbc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd8f7abdd780ebe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:13.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.750+0000 7fdfd0f6f640 1 -- 172.21.15.97:0/996790477 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc034070 con 0x7fdfcc06f3f0 2024-03-29T04:08:13.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.750+0000 7fdfd0f6f640 1 -- 172.21.15.97:0/996790477 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfbc030610 con 0x7fdfcc06f3f0 2024-03-29T04:08:13.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.750+0000 7fdfd0f6f640 1 -- 172.21.15.97:0/996790477 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0309e0 con 0x7fdfcc06f3f0 2024-03-29T04:08:13.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.750+0000 7fdfd41d5640 1 -- 172.21.15.97:0/996790477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc06f3f0 msgr2=0x7fdfcc072b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.750+0000 7fdfd41d5640 1 --2- 172.21.15.97:0/996790477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc06f3f0 0x7fdfcc072b80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0097a0 tx=0x7fdfbc02b960 comp rx=0 tx=0).stop 2024-03-29T04:08:13.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.750+0000 7fdfd41d5640 1 -- 172.21.15.97:0/996790477 shutdown_connections 2024-03-29T04:08:13.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.750+0000 7fdfd41d5640 1 --2- 172.21.15.97:0/996790477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc06f3f0 0x7fdfcc072b80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.750+0000 7fdfd41d5640 1 -- 172.21.15.97:0/996790477 >> 172.21.15.97:0/996790477 conn(0x7fdfcc06b510 msgr2=0x7fdfcc06b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:13.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.750+0000 7fdfd41d5640 1 -- 172.21.15.97:0/996790477 shutdown_connections 2024-03-29T04:08:13.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.750+0000 7fdfd41d5640 1 -- 172.21.15.97:0/996790477 wait complete. 2024-03-29T04:08:13.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.751+0000 7fdfd41d5640 1 Processor -- start 2024-03-29T04:08:13.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.752+0000 7fdfd41d5640 1 -- start start 2024-03-29T04:08:13.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.752+0000 7fdfd41d5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc06f3f0 0x7fdfcc15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:13.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.752+0000 7fdfd41d5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc0730c0 0x7fdfcc14f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:13.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.752+0000 7fdfd41d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc15b370 0x7fdfcc15e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:13.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.752+0000 7fdfd41d5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc073550 con 0x7fdfcc15b370 2024-03-29T04:08:13.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.752+0000 7fdfd41d5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc072660 con 0x7fdfcc06f3f0 2024-03-29T04:08:13.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.752+0000 7fdfd41d5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc15b750 con 0x7fdfcc0730c0 2024-03-29T04:08:13.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.752+0000 7fdfd2772640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc15b370 0x7fdfcc15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:13.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.752+0000 7fdfd2772640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc15b370 0x7fdfcc15e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34674/0 (socket says 172.21.15.97:34674) 2024-03-29T04:08:13.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.752+0000 7fdfd2772640 1 -- 172.21.15.97:0/4248737642 learned_addr learned my addr 172.21.15.97:0/4248737642 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:13.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.752+0000 7fdfd1770640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc0730c0 0x7fdfcc14f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.752+0000 7fdfd1f71640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc06f3f0 0x7fdfcc15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.753+0000 7fdfd2772640 1 -- 172.21.15.97:0/4248737642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc0730c0 msgr2=0x7fdfcc14f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.753+0000 7fdfd2772640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc0730c0 0x7fdfcc14f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.753+0000 7fdfd2772640 1 -- 172.21.15.97:0/4248737642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc06f3f0 msgr2=0x7fdfcc15cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.753+0000 7fdfd2772640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc06f3f0 0x7fdfcc15cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.753+0000 7fdfd2772640 1 -- 172.21.15.97:0/4248737642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc15bb20 con 0x7fdfcc15b370 2024-03-29T04:08:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.753+0000 7fdfd2772640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc15b370 0x7fdfcc15e270 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800c440 tx=0x7fdfc800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:13.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.753+0000 7fdfc27fc640 1 -- 172.21.15.97:0/4248737642 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc80074e0 con 0x7fdfcc15b370 2024-03-29T04:08:13.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.753+0000 7fdfc27fc640 1 -- 172.21.15.97:0/4248737642 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc80047e0 con 0x7fdfcc15b370 2024-03-29T04:08:13.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.753+0000 7fdfd41d5640 1 -- 172.21.15.97:0/4248737642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc15be00 con 0x7fdfcc15b370 2024-03-29T04:08:13.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.753+0000 7fdfc27fc640 1 -- 172.21.15.97:0/4248737642 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8004bd0 con 0x7fdfcc15b370 2024-03-29T04:08:13.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.753+0000 7fdfd41d5640 1 -- 172.21.15.97:0/4248737642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc15c3c0 con 0x7fdfcc15b370 2024-03-29T04:08:13.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.754+0000 7fdfd41d5640 1 -- 172.21.15.97:0/4248737642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfcc0ad360 con 0x7fdfcc15b370 2024-03-29T04:08:13.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.754+0000 7fdfc27fc640 1 -- 172.21.15.97:0/4248737642 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdfc8011070 con 0x7fdfcc15b370 2024-03-29T04:08:13.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.755+0000 7fdfc27fc640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfb006c530 0x7fdfb006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:13.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.755+0000 7fdfd1f71640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfb006c530 0x7fdfb006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:13.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.755+0000 7fdfc27fc640 1 -- 172.21.15.97:0/4248737642 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdfc808b120 con 0x7fdfcc15b370 2024-03-29T04:08:13.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.755+0000 7fdfd1f71640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfb006c530 0x7fdfb006e9f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40012e0 tx=0x7fdfc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:13.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.759+0000 7fdfc27fc640 1 -- 172.21.15.97:0/4248737642 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdfcc0ad360 con 0x7fdfcc15b370 2024-03-29T04:08:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.934+0000 7fdfd41d5640 1 -- 172.21.15.97:0/4248737642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfcc068640 con 0x7fdfcc15b370 2024-03-29T04:08:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.935+0000 7fdfc27fc640 1 -- 172.21.15.97:0/4248737642 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fdfc80590f0 con 0x7fdfcc15b370 2024-03-29T04:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:13.915 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.937+0000 7fdfd41d5640 1 -- 172.21.15.97:0/4248737642 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfb006c530 msgr2=0x7fdfb006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.937+0000 7fdfd41d5640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfb006c530 0x7fdfb006e9f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40012e0 tx=0x7fdfc4009040 comp rx=0 tx=0).stop 2024-03-29T04:08:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.937+0000 7fdfd41d5640 1 -- 172.21.15.97:0/4248737642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc15b370 msgr2=0x7fdfcc15e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.937+0000 7fdfd41d5640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc15b370 0x7fdfcc15e270 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800c440 tx=0x7fdfc800c900 comp rx=0 tx=0).stop 2024-03-29T04:08:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.937+0000 7fdfd41d5640 1 -- 172.21.15.97:0/4248737642 shutdown_connections 2024-03-29T04:08:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.937+0000 7fdfd41d5640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdfb006c530 0x7fdfb006e9f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.937+0000 7fdfd41d5640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfcc15b370 0x7fdfcc15e270 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.937+0000 7fdfd41d5640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdfcc0730c0 0x7fdfcc14f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.937+0000 7fdfd41d5640 1 --2- 172.21.15.97:0/4248737642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfcc06f3f0 0x7fdfcc15cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.937+0000 7fdfd41d5640 1 -- 172.21.15.97:0/4248737642 >> 172.21.15.97:0/4248737642 conn(0x7fdfcc06b510 msgr2=0x7fdfcc1577c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.937+0000 7fdfd41d5640 1 -- 172.21.15.97:0/4248737642 shutdown_connections 2024-03-29T04:08:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:13.937+0000 7fdfd41d5640 1 -- 172.21.15.97:0/4248737642 wait complete. 2024-03-29T04:08:13.928 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:07:38.618704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:07:39.724339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:08:13.928 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:08:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.051+0000 7fd608b90640 1 Processor -- start 2024-03-29T04:08:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.051+0000 7fd608b90640 1 -- start start 2024-03-29T04:08:14.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.052+0000 7fd608b90640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd6040d16a0 0x7fd6040d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.052+0000 7fd608b90640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd60415ed00 con 0x7fd6040cfde0 2024-03-29T04:08:14.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.052+0000 7fd608b90640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6040d21e0 con 0x7fd60415e920 2024-03-29T04:08:14.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.052+0000 7fd608b90640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fd6040d2480 con 0x7fd6040d16a0 2024-03-29T04:08:14.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.053+0000 7fd601d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd6040cfde0 0x7fd6040d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35424/0 (socket says 172.21.15.97:35424) 2024-03-29T04:08:14.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.053+0000 7fd601d9b640 1 -- 172.21.15.97:0/1733211963 learned_addr learned my addr 172.21.15.97:0/1733211963 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:14.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.053+0000 7fd602d9d640 1 --2- 172.21.15.97:0/1733211963 >> v2:172.21.15.154:3300/0 conn(0x7fd6040d16a0 0x7fd6040d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.053+0000 7fd60159a640 1 -- 172.21.15.97:0/1733211963 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1382790347 0 0) 0x7fd60415ed00 con 0x7fd6040cfde0 2024-03-29T04:08:14.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.053+0000 7fd60159a640 1 -- 172.21.15.97:0/1733211963 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5ec003520 con 0x7fd6040cfde0 2024-03-29T04:08:14.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.053+0000 7fd60159a640 1 -- 172.21.15.97:0/1733211963 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1516913889 0 0) 0x7fd6040d21e0 con 0x7fd60415e920 2024-03-29T04:08:14.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.053+0000 7fd60159a640 1 -- 172.21.15.97:0/1733211963 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd60415ed00 con 0x7fd60415e920 2024-03-29T04:08:14.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.053+0000 7fd602d9d640 1 -- 172.21.15.97:0/1733211963 >> v1:172.21.15.154:6789/0 conn(0x7fd60415e920 legacy=0x7fd6040d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.053+0000 7fd602d9d640 1 -- 172.21.15.97:0/1733211963 >> v1:172.21.15.97:6789/0 conn(0x7fd6040cfde0 legacy=0x7fd6040d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.053+0000 7fd602d9d640 1 -- 172.21.15.97:0/1733211963 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6040d2760 con 0x7fd6040d16a0 2024-03-29T04:08:14.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.054+0000 7fd602d9d640 1 --2- 172.21.15.97:0/1733211963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6040d16a0 0x7fd6040d0240 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0097a0 tx=0x7fd5fc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=603bd3966e929650 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:14.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.054+0000 7fd60159a640 1 -- 172.21.15.97:0/1733211963 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc034070 con 0x7fd6040d16a0 2024-03-29T04:08:14.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.054+0000 7fd60159a640 1 -- 172.21.15.97:0/1733211963 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5fc030610 con 0x7fd6040d16a0 2024-03-29T04:08:14.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.054+0000 7fd60159a640 1 -- 172.21.15.97:0/1733211963 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0309e0 con 0x7fd6040d16a0 2024-03-29T04:08:14.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.054+0000 7fd608b90640 1 -- 172.21.15.97:0/1733211963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6040d16a0 msgr2=0x7fd6040d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.054+0000 7fd608b90640 1 --2- 172.21.15.97:0/1733211963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6040d16a0 0x7fd6040d0240 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0097a0 tx=0x7fd5fc02b960 comp rx=0 tx=0).stop 2024-03-29T04:08:14.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.054+0000 7fd608b90640 1 -- 172.21.15.97:0/1733211963 shutdown_connections 2024-03-29T04:08:14.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.054+0000 7fd608b90640 1 --2- 172.21.15.97:0/1733211963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6040d16a0 0x7fd6040d0240 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.054+0000 7fd608b90640 1 -- 172.21.15.97:0/1733211963 >> 172.21.15.97:0/1733211963 conn(0x7fd60408bfc0 msgr2=0x7fd6040d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:14.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.055+0000 7fd608b90640 1 -- 172.21.15.97:0/1733211963 shutdown_connections 2024-03-29T04:08:14.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.055+0000 7fd608b90640 1 -- 172.21.15.97:0/1733211963 wait complete. 2024-03-29T04:08:14.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.056+0000 7fd608b90640 1 Processor -- start 2024-03-29T04:08:14.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.056+0000 7fd608b90640 1 -- start start 2024-03-29T04:08:14.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.056+0000 7fd608b90640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6040cfde0 0x7fd6040cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.056+0000 7fd608b90640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6040d16a0 0x7fd6040b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.056+0000 7fd608b90640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd60415e920 0x7fd60414ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.056+0000 7fd608b90640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd6040d4ef0 con 0x7fd6040cfde0 2024-03-29T04:08:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.056+0000 7fd608b90640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd6040d2d20 con 0x7fd60415e920 2024-03-29T04:08:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.056+0000 7fd608b90640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd6040d5070 con 0x7fd6040d16a0 2024-03-29T04:08:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd60259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6040cfde0 0x7fd6040cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd60259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6040cfde0 0x7fd6040cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34676/0 (socket says 172.21.15.97:34676) 2024-03-29T04:08:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd60259c640 1 -- 172.21.15.97:0/1672741893 learned_addr learned my addr 172.21.15.97:0/1672741893 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd601d9b640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6040d16a0 0x7fd6040b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd602d9d640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd60415e920 0x7fd60414ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd60259c640 1 -- 172.21.15.97:0/1672741893 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6040d16a0 msgr2=0x7fd6040b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd60259c640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6040d16a0 0x7fd6040b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd60259c640 1 -- 172.21.15.97:0/1672741893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd60415e920 msgr2=0x7fd60414ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd60259c640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd60415e920 0x7fd60414ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd60259c640 1 -- 172.21.15.97:0/1672741893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60414f120 con 0x7fd6040cfde0 2024-03-29T04:08:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd60259c640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6040cfde0 0x7fd6040cb6e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400c7d0 tx=0x7fd5f400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd5f2ffd640 1 -- 172.21.15.97:0/1672741893 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4003b70 con 0x7fd6040cfde0 2024-03-29T04:08:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd5f2ffd640 1 -- 172.21.15.97:0/1672741893 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f4003d30 con 0x7fd6040cfde0 2024-03-29T04:08:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd5f2ffd640 1 -- 172.21.15.97:0/1672741893 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400aa30 con 0x7fd6040cfde0 2024-03-29T04:08:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd608b90640 1 -- 172.21.15.97:0/1672741893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6040cbd70 con 0x7fd6040cfde0 2024-03-29T04:08:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.057+0000 7fd608b90640 1 -- 172.21.15.97:0/1672741893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6040cc330 con 0x7fd6040cfde0 2024-03-29T04:08:14.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.058+0000 7fd608b90640 1 -- 172.21.15.97:0/1672741893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6040d0240 con 0x7fd6040cfde0 2024-03-29T04:08:14.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.059+0000 7fd5f2ffd640 1 -- 172.21.15.97:0/1672741893 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd5f400abd0 con 0x7fd6040cfde0 2024-03-29T04:08:14.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.059+0000 7fd5f2ffd640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd5e406c530 0x7fd5e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.059+0000 7fd601d9b640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd5e406c530 0x7fd5e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.059+0000 7fd5f2ffd640 1 -- 172.21.15.97:0/1672741893 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd5f408b420 con 0x7fd6040cfde0 2024-03-29T04:08:14.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.060+0000 7fd601d9b640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd5e406c530 0x7fd5e406e9f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd6040b4c50 tx=0x7fd5f8007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:14.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.064+0000 7fd5f2ffd640 1 -- 172.21.15.97:0/1672741893 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd6040d0240 con 0x7fd6040cfde0 2024-03-29T04:08:14.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.238+0000 7fd608b90640 1 -- 172.21.15.97:0/1672741893 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd604067790 con 0x7fd6040cfde0 2024-03-29T04:08:14.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.239+0000 7fd5f2ffd640 1 -- 172.21.15.97:0/1672741893 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fd604067790 con 0x7fd6040cfde0 2024-03-29T04:08:14.219 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:08:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.241+0000 7fd608b90640 1 -- 172.21.15.97:0/1672741893 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd5e406c530 msgr2=0x7fd5e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.241+0000 7fd608b90640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd5e406c530 0x7fd5e406e9f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd6040b4c50 tx=0x7fd5f8007d30 comp rx=0 tx=0).stop 2024-03-29T04:08:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.241+0000 7fd608b90640 1 -- 172.21.15.97:0/1672741893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6040cfde0 msgr2=0x7fd6040cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.241+0000 7fd608b90640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6040cfde0 0x7fd6040cb6e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400c7d0 tx=0x7fd5f400cc90 comp rx=0 tx=0).stop 2024-03-29T04:08:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.241+0000 7fd608b90640 1 -- 172.21.15.97:0/1672741893 shutdown_connections 2024-03-29T04:08:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.241+0000 7fd608b90640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd5e406c530 0x7fd5e406e9f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.241+0000 7fd608b90640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd60415e920 0x7fd60414ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.241+0000 7fd608b90640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd6040d16a0 0x7fd6040b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.241+0000 7fd608b90640 1 --2- 172.21.15.97:0/1672741893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6040cfde0 0x7fd6040cb6e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.241+0000 7fd608b90640 1 -- 172.21.15.97:0/1672741893 >> 172.21.15.97:0/1672741893 conn(0x7fd60408bfc0 msgr2=0x7fd6040d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:14.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.241+0000 7fd608b90640 1 -- 172.21.15.97:0/1672741893 shutdown_connections 2024-03-29T04:08:14.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.241+0000 7fd608b90640 1 -- 172.21.15.97:0/1672741893 wait complete. 2024-03-29T04:08:14.231 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:08:14.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.357+0000 7f7ef770d640 1 Processor -- start 2024-03-29T04:08:14.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.357+0000 7f7ef770d640 1 -- start start 2024-03-29T04:08:14.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef770d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ef00ca950 0x7f7ef00b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef770d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ef00b26f0 0x7f7ef015be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef770d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef00cade0 con 0x7f7ef015b3d0 2024-03-29T04:08:14.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef770d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef00b2b60 con 0x7f7ef00ca950 2024-03-29T04:08:14.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef770d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef00c6240 con 0x7f7ef00b26f0 2024-03-29T04:08:14.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef4ca8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ef00ca950 0x7f7ef00b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef54a9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7ef015b3d0 0x7f7ef00b49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35428/0 (socket says 172.21.15.97:35428) 2024-03-29T04:08:14.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef4ca8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ef00ca950 0x7f7ef00b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34680/0 (socket says 172.21.15.97:34680) 2024-03-29T04:08:14.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef54a9640 1 -- 172.21.15.97:0/3309401116 learned_addr learned my addr 172.21.15.97:0/3309401116 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:14.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef5caa640 1 --2- 172.21.15.97:0/3309401116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ef00b26f0 0x7f7ef015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef5caa640 1 -- 172.21.15.97:0/3309401116 >> v2:172.21.15.97:3300/0 conn(0x7f7ef00ca950 msgr2=0x7f7ef00b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:14.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef5caa640 1 --2- 172.21.15.97:0/3309401116 >> v2:172.21.15.97:3300/0 conn(0x7f7ef00ca950 0x7f7ef00b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef5caa640 1 -- 172.21.15.97:0/3309401116 >> v1:172.21.15.97:6789/0 conn(0x7f7ef015b3d0 legacy=0x7f7ef00b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef5caa640 1 -- 172.21.15.97:0/3309401116 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef00c6470 con 0x7f7ef00b26f0 2024-03-29T04:08:14.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ee3fff640 1 -- 172.21.15.97:0/3309401116 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 196393732 0 0) 0x7f7ef00cade0 con 0x7f7ef015b3d0 2024-03-29T04:08:14.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.358+0000 7f7ef4ca8640 1 --2- 172.21.15.97:0/3309401116 >> v2:172.21.15.97:3300/0 conn(0x7f7ef00ca950 0x7f7ef00b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:14.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.359+0000 7f7ef5caa640 1 --2- 172.21.15.97:0/3309401116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ef00b26f0 0x7f7ef015be50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009d90 tx=0x7f7ee802beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=626d1eed4a5fe4b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:14.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.359+0000 7f7ee3fff640 1 -- 172.21.15.97:0/3309401116 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8039070 con 0x7f7ef00b26f0 2024-03-29T04:08:14.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.359+0000 7f7ee3fff640 1 -- 172.21.15.97:0/3309401116 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee802cc60 con 0x7f7ef00b26f0 2024-03-29T04:08:14.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.359+0000 7f7ee3fff640 1 -- 172.21.15.97:0/3309401116 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8034660 con 0x7f7ef00b26f0 2024-03-29T04:08:14.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.360+0000 7f7ef770d640 1 -- 172.21.15.97:0/3309401116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ef00b26f0 msgr2=0x7f7ef015be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.360+0000 7f7ef770d640 1 --2- 172.21.15.97:0/3309401116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ef00b26f0 0x7f7ef015be50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009d90 tx=0x7f7ee802beb0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.360+0000 7f7ef770d640 1 -- 172.21.15.97:0/3309401116 shutdown_connections 2024-03-29T04:08:14.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.360+0000 7f7ef770d640 1 --2- 172.21.15.97:0/3309401116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ef00b26f0 0x7f7ef015be50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.360+0000 7f7ef770d640 1 --2- 172.21.15.97:0/3309401116 >> v2:172.21.15.97:3300/0 conn(0x7f7ef00ca950 0x7f7ef00b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.360+0000 7f7ef770d640 1 -- 172.21.15.97:0/3309401116 >> 172.21.15.97:0/3309401116 conn(0x7f7ef00d0450 msgr2=0x7f7ef00d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:14.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.360+0000 7f7ef770d640 1 -- 172.21.15.97:0/3309401116 shutdown_connections 2024-03-29T04:08:14.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.360+0000 7f7ef770d640 1 -- 172.21.15.97:0/3309401116 wait complete. 2024-03-29T04:08:14.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.361+0000 7f7ef770d640 1 Processor -- start 2024-03-29T04:08:14.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.361+0000 7f7ef770d640 1 -- start start 2024-03-29T04:08:14.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef770d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef00b26f0 0x7f7ef00c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef770d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ef00ca950 0x7f7ef01e2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef770d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ef015b3d0 0x7f7ef01e3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef770d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef00c6a30 con 0x7f7ef00b26f0 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef770d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef00c7080 con 0x7f7ef015b3d0 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef770d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef00c6d10 con 0x7f7ef00ca950 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef54a9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef00b26f0 0x7f7ef00c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef54a9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef00b26f0 0x7f7ef00c72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34684/0 (socket says 172.21.15.97:34684) 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef54a9640 1 -- 172.21.15.97:0/988956313 learned_addr learned my addr 172.21.15.97:0/988956313 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef4ca8640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ef00ca950 0x7f7ef01e2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef5caa640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ef015b3d0 0x7f7ef01e3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef54a9640 1 -- 172.21.15.97:0/988956313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ef00ca950 msgr2=0x7f7ef01e2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef54a9640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ef00ca950 0x7f7ef01e2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef54a9640 1 -- 172.21.15.97:0/988956313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ef015b3d0 msgr2=0x7f7ef01e3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef54a9640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ef015b3d0 0x7f7ef01e3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef54a9640 1 -- 172.21.15.97:0/988956313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef00ae840 con 0x7f7ef00b26f0 2024-03-29T04:08:14.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.362+0000 7f7ef54a9640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef00b26f0 0x7f7ef00c72b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00c430 tx=0x7f7eec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:14.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.363+0000 7f7ee1ffb640 1 -- 172.21.15.97:0/988956313 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec017070 con 0x7f7ef00b26f0 2024-03-29T04:08:14.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.363+0000 7f7ee1ffb640 1 -- 172.21.15.97:0/988956313 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec005ce0 con 0x7f7ef00b26f0 2024-03-29T04:08:14.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.363+0000 7f7ef770d640 1 -- 172.21.15.97:0/988956313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef0071a20 con 0x7f7ef00b26f0 2024-03-29T04:08:14.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.363+0000 7f7ee1ffb640 1 -- 172.21.15.97:0/988956313 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec007310 con 0x7f7ef00b26f0 2024-03-29T04:08:14.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.363+0000 7f7ef770d640 1 -- 172.21.15.97:0/988956313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef01e5d50 con 0x7f7ef00b26f0 2024-03-29T04:08:14.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.364+0000 7f7ef770d640 1 -- 172.21.15.97:0/988956313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef009e520 con 0x7f7ef00b26f0 2024-03-29T04:08:14.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.364+0000 7f7ee1ffb640 1 -- 172.21.15.97:0/988956313 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7eec01a070 con 0x7f7ef00b26f0 2024-03-29T04:08:14.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.365+0000 7f7ee1ffb640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ed406c560 0x7f7ed406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.365+0000 7f7ee1ffb640 1 -- 172.21.15.97:0/988956313 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7eec08a980 con 0x7f7ef00b26f0 2024-03-29T04:08:14.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.365+0000 7f7ef4ca8640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ed406c560 0x7f7ed406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.365+0000 7f7ef4ca8640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ed406c560 0x7f7ed406ea20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009560 tx=0x7f7ee4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:14.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.369+0000 7f7ee1ffb640 1 -- 172.21.15.97:0/988956313 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7eec0589e0 con 0x7f7ef00b26f0 2024-03-29T04:08:14.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.549+0000 7f7ef770d640 1 -- 172.21.15.97:0/988956313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7ef004f9d0 con 0x7f7ef00b26f0 2024-03-29T04:08:14.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.549+0000 7f7ee1ffb640 1 -- 172.21.15.97:0/988956313 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f7eec05cce0 con 0x7f7ef00b26f0 2024-03-29T04:08:14.529 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:08:14.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.551+0000 7f7ef770d640 1 -- 172.21.15.97:0/988956313 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ed406c560 msgr2=0x7f7ed406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.551+0000 7f7ef770d640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ed406c560 0x7f7ed406ea20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009560 tx=0x7f7ee4009190 comp rx=0 tx=0).stop 2024-03-29T04:08:14.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.551+0000 7f7ef770d640 1 -- 172.21.15.97:0/988956313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef00b26f0 msgr2=0x7f7ef00c72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.551+0000 7f7ef770d640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef00b26f0 0x7f7ef00c72b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00c430 tx=0x7f7eec00c8f0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.551+0000 7f7ef770d640 1 -- 172.21.15.97:0/988956313 shutdown_connections 2024-03-29T04:08:14.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.551+0000 7f7ef770d640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7ed406c560 0x7f7ed406ea20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.551+0000 7f7ef770d640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ef015b3d0 0x7f7ef01e3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.551+0000 7f7ef770d640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ef00ca950 0x7f7ef01e2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.551+0000 7f7ef770d640 1 --2- 172.21.15.97:0/988956313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef00b26f0 0x7f7ef00c72b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.551+0000 7f7ef770d640 1 -- 172.21.15.97:0/988956313 >> 172.21.15.97:0/988956313 conn(0x7f7ef00d0450 msgr2=0x7f7ef015a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:14.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.552+0000 7f7ef770d640 1 -- 172.21.15.97:0/988956313 shutdown_connections 2024-03-29T04:08:14.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.552+0000 7f7ef770d640 1 -- 172.21.15.97:0/988956313 wait complete. 2024-03-29T04:08:14.542 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:08:14.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.670+0000 7f703b174640 1 Processor -- start 2024-03-29T04:08:14.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.670+0000 7f703b174640 1 -- start start 2024-03-29T04:08:14.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f703b174640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f703415b8e0 0x7f70340cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f703b174640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f70340c7120 0x7f70340cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f703b174640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70340b73f0 con 0x7f70340d0450 2024-03-29T04:08:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f703b174640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f70340d09c0 con 0x7f70340c7120 2024-03-29T04:08:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f703b174640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f70340cba90 con 0x7f703415b8e0 2024-03-29T04:08:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f7039711640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f70340c7120 0x7f70340cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f7033fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f70340d0450 0x7f70340c6a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35430/0 (socket says 172.21.15.97:35430) 2024-03-29T04:08:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f7033fff640 1 -- 172.21.15.97:0/1239505483 learned_addr learned my addr 172.21.15.97:0/1239505483 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f7038f10640 1 --2- 172.21.15.97:0/1239505483 >> v2:172.21.15.154:3300/0 conn(0x7f703415b8e0 0x7f70340cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f7039711640 1 -- 172.21.15.97:0/1239505483 >> v2:172.21.15.154:3300/0 conn(0x7f703415b8e0 msgr2=0x7f70340cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f7039711640 1 --2- 172.21.15.97:0/1239505483 >> v2:172.21.15.154:3300/0 conn(0x7f703415b8e0 0x7f70340cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f7039711640 1 -- 172.21.15.97:0/1239505483 >> v1:172.21.15.97:6789/0 conn(0x7f70340d0450 legacy=0x7f70340c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f70337fe640 1 -- 172.21.15.97:0/1239505483 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3688025425 0 0) 0x7f70340b73f0 con 0x7f70340d0450 2024-03-29T04:08:14.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.671+0000 7f7039711640 1 -- 172.21.15.97:0/1239505483 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70340cbd70 con 0x7f70340c7120 2024-03-29T04:08:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.672+0000 7f7039711640 1 --2- 172.21.15.97:0/1239505483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70340c7120 0x7f70340cc850 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7034073b50 tx=0x7f702402e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e610b4210d0b9f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.673+0000 7f70337fe640 1 -- 172.21.15.97:0/1239505483 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702403b070 con 0x7f70340c7120 2024-03-29T04:08:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.673+0000 7f70337fe640 1 -- 172.21.15.97:0/1239505483 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702402ee50 con 0x7f70340c7120 2024-03-29T04:08:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.673+0000 7f70337fe640 1 -- 172.21.15.97:0/1239505483 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024036660 con 0x7f70340c7120 2024-03-29T04:08:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.673+0000 7f703b174640 1 -- 172.21.15.97:0/1239505483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70340c7120 msgr2=0x7f70340cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.673+0000 7f703b174640 1 --2- 172.21.15.97:0/1239505483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70340c7120 0x7f70340cc850 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7034073b50 tx=0x7f702402e0a0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.673+0000 7f703b174640 1 -- 172.21.15.97:0/1239505483 shutdown_connections 2024-03-29T04:08:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.673+0000 7f703b174640 1 --2- 172.21.15.97:0/1239505483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70340c7120 0x7f70340cc850 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.673+0000 7f703b174640 1 --2- 172.21.15.97:0/1239505483 >> v2:172.21.15.154:3300/0 conn(0x7f703415b8e0 0x7f70340cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.673+0000 7f703b174640 1 -- 172.21.15.97:0/1239505483 >> 172.21.15.97:0/1239505483 conn(0x7f70340b2fd0 msgr2=0x7f70340b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.673+0000 7f703b174640 1 -- 172.21.15.97:0/1239505483 shutdown_connections 2024-03-29T04:08:14.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.673+0000 7f703b174640 1 -- 172.21.15.97:0/1239505483 wait complete. 2024-03-29T04:08:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.675+0000 7f703b174640 1 Processor -- start 2024-03-29T04:08:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.675+0000 7f703b174640 1 -- start start 2024-03-29T04:08:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.675+0000 7f703b174640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70340c7120 0x7f703414ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.675+0000 7f703b174640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70340d0450 0x7f703415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.675+0000 7f703b174640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f703415b8e0 0x7f703415de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.675+0000 7f703b174640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f70340b42d0 con 0x7f703415b8e0 2024-03-29T04:08:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.675+0000 7f703b174640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f70340cc330 con 0x7f70340c7120 2024-03-29T04:08:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.675+0000 7f703b174640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f70340c9330 con 0x7f70340d0450 2024-03-29T04:08:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f7039711640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f703415b8e0 0x7f703415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f7038f10640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70340c7120 0x7f703414ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f7039711640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f703415b8e0 0x7f703415de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34706/0 (socket says 172.21.15.97:34706) 2024-03-29T04:08:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f7033fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70340d0450 0x7f703415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f7039711640 1 -- 172.21.15.97:0/2719252835 learned_addr learned my addr 172.21.15.97:0/2719252835 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:14.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f7039711640 1 -- 172.21.15.97:0/2719252835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70340d0450 msgr2=0x7f703415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f7039711640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70340d0450 0x7f703415d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f7039711640 1 -- 172.21.15.97:0/2719252835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70340c7120 msgr2=0x7f703414ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f7039711640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70340c7120 0x7f703414ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f7039711640 1 -- 172.21.15.97:0/2719252835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70340c9610 con 0x7f703415b8e0 2024-03-29T04:08:14.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f7038f10640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70340c7120 0x7f703414ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:14.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f7033fff640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70340d0450 0x7f703415d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:14.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f7039711640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f703415b8e0 0x7f703415de30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f702402e6b0 tx=0x7f70240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:14.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f70317fa640 1 -- 172.21.15.97:0/2719252835 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024046020 con 0x7f703415b8e0 2024-03-29T04:08:14.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f70317fa640 1 -- 172.21.15.97:0/2719252835 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702403c040 con 0x7f703415b8e0 2024-03-29T04:08:14.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f703b174640 1 -- 172.21.15.97:0/2719252835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70340c98f0 con 0x7f703415b8e0 2024-03-29T04:08:14.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f70317fa640 1 -- 172.21.15.97:0/2719252835 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702403b070 con 0x7f703415b8e0 2024-03-29T04:08:14.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.676+0000 7f703b174640 1 -- 172.21.15.97:0/2719252835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703414b260 con 0x7f703415b8e0 2024-03-29T04:08:14.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.677+0000 7f703b174640 1 -- 172.21.15.97:0/2719252835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703409f630 con 0x7f703415b8e0 2024-03-29T04:08:14.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.678+0000 7f70317fa640 1 -- 172.21.15.97:0/2719252835 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7024049020 con 0x7f703415b8e0 2024-03-29T04:08:14.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.679+0000 7f70317fa640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f702006c530 0x7f702006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.679+0000 7f7038f10640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f702006c530 0x7f702006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.679+0000 7f70317fa640 1 -- 172.21.15.97:0/2719252835 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f70240b0130 con 0x7f703415b8e0 2024-03-29T04:08:14.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.679+0000 7f7038f10640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f702006c530 0x7f702006e9f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f702c009fd0 tx=0x7f702c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:14.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.685+0000 7f70317fa640 1 -- 172.21.15.97:0/2719252835 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7024050020 con 0x7f703415b8e0 2024-03-29T04:08:14.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.867+0000 7f703b174640 1 -- 172.21.15.97:0/2719252835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f703404f9d0 con 0x7f703415b8e0 2024-03-29T04:08:14.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.868+0000 7f70317fa640 1 -- 172.21.15.97:0/2719252835 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f702407e100 con 0x7f703415b8e0 2024-03-29T04:08:14.848 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:08:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.870+0000 7f703b174640 1 -- 172.21.15.97:0/2719252835 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f702006c530 msgr2=0x7f702006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.870+0000 7f703b174640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f702006c530 0x7f702006e9f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f702c009fd0 tx=0x7f702c00b040 comp rx=0 tx=0).stop 2024-03-29T04:08:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.870+0000 7f703b174640 1 -- 172.21.15.97:0/2719252835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f703415b8e0 msgr2=0x7f703415de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.870+0000 7f703b174640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f703415b8e0 0x7f703415de30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f702402e6b0 tx=0x7f70240046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.870+0000 7f703b174640 1 -- 172.21.15.97:0/2719252835 shutdown_connections 2024-03-29T04:08:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.870+0000 7f703b174640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f702006c530 0x7f702006e9f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.870+0000 7f703b174640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f703415b8e0 0x7f703415de30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.870+0000 7f703b174640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f70340d0450 0x7f703415d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.870+0000 7f703b174640 1 --2- 172.21.15.97:0/2719252835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70340c7120 0x7f703414ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.870+0000 7f703b174640 1 -- 172.21.15.97:0/2719252835 >> 172.21.15.97:0/2719252835 conn(0x7f70340b2fd0 msgr2=0x7f70340d0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.870+0000 7f703b174640 1 -- 172.21.15.97:0/2719252835 shutdown_connections 2024-03-29T04:08:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.870+0000 7f703b174640 1 -- 172.21.15.97:0/2719252835 wait complete. 2024-03-29T04:08:14.861 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:08:14.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.985+0000 7f60bc300640 1 Processor -- start 2024-03-29T04:08:14.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.986+0000 7f60bc300640 1 -- start start 2024-03-29T04:08:14.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.986+0000 7f60bc300640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f60b415e780 0x7f60b40cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.986+0000 7f60bc300640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60b40cfe70 con 0x7f60b40ba000 2024-03-29T04:08:14.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.986+0000 7f60bc300640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60b40b7f50 con 0x7f60b415eb60 2024-03-29T04:08:14.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.986+0000 7f60bc300640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f60b40b8230 con 0x7f60b415e780 2024-03-29T04:08:14.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.987+0000 7f60ba09c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f60b415e780 0x7f60b40cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.987+0000 7f60ba89d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f60b40ba000 0x7f60b40d2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35434/0 (socket says 172.21.15.97:35434) 2024-03-29T04:08:14.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.987+0000 7f60ba09c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f60b415e780 0x7f60b40cebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34708/0 (socket says 172.21.15.97:34708) 2024-03-29T04:08:14.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.987+0000 7f60ba89d640 1 -- 172.21.15.97:0/3277088664 learned_addr learned my addr 172.21.15.97:0/3277088664 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:14.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.987+0000 7f60ba09c640 1 -- 172.21.15.97:0/3277088664 >> v1:172.21.15.154:6789/0 conn(0x7f60b415eb60 legacy=0x7f60b40cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.987+0000 7f60b909a640 1 -- 172.21.15.97:0/3277088664 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595095268 0 0) 0x7f60b40b7f50 con 0x7f60b415eb60 2024-03-29T04:08:14.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.987+0000 7f60ba09c640 1 -- 172.21.15.97:0/3277088664 >> v1:172.21.15.97:6789/0 conn(0x7f60b40ba000 legacy=0x7f60b40d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.987+0000 7f60ba09c640 1 -- 172.21.15.97:0/3277088664 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b40b8510 con 0x7f60b415e780 2024-03-29T04:08:14.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.988+0000 7f60ba09c640 1 --2- 172.21.15.97:0/3277088664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60b415e780 0x7f60b40cebc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f60b00097a0 tx=0x7f60b002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12445455bee8b740 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:14.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.988+0000 7f60b909a640 1 -- 172.21.15.97:0/3277088664 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0037070 con 0x7f60b415e780 2024-03-29T04:08:14.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.988+0000 7f60b909a640 1 -- 172.21.15.97:0/3277088664 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b0030710 con 0x7f60b415e780 2024-03-29T04:08:14.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.988+0000 7f60b909a640 1 -- 172.21.15.97:0/3277088664 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0030ae0 con 0x7f60b415e780 2024-03-29T04:08:14.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.989+0000 7f60bc300640 1 -- 172.21.15.97:0/3277088664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60b415e780 msgr2=0x7f60b40cebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.989+0000 7f60bc300640 1 --2- 172.21.15.97:0/3277088664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60b415e780 0x7f60b40cebc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f60b00097a0 tx=0x7f60b002b790 comp rx=0 tx=0).stop 2024-03-29T04:08:14.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.989+0000 7f60bc300640 1 -- 172.21.15.97:0/3277088664 shutdown_connections 2024-03-29T04:08:14.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.989+0000 7f60bc300640 1 --2- 172.21.15.97:0/3277088664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60b415e780 0x7f60b40cebc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.989+0000 7f60bc300640 1 -- 172.21.15.97:0/3277088664 >> 172.21.15.97:0/3277088664 conn(0x7f60b408bfc0 msgr2=0x7f60b40d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:14.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.989+0000 7f60bc300640 1 -- 172.21.15.97:0/3277088664 shutdown_connections 2024-03-29T04:08:14.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.989+0000 7f60bc300640 1 -- 172.21.15.97:0/3277088664 wait complete. 2024-03-29T04:08:14.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.990+0000 7f60bc300640 1 Processor -- start 2024-03-29T04:08:14.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.990+0000 7f60bc300640 1 -- start start 2024-03-29T04:08:14.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60bc300640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60b40ba000 0x7f60b4161390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60bc300640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60b415e780 0x7f60b4158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60bc300640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60b4158ab0 0x7f60b41f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60bc300640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f60b40cf810 con 0x7f60b4158ab0 2024-03-29T04:08:14.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60bc300640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f60b40b8ad0 con 0x7f60b40ba000 2024-03-29T04:08:14.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60bc300640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f60b40cf990 con 0x7f60b415e780 2024-03-29T04:08:14.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60ba89d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60b4158ab0 0x7f60b41f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60ba89d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60b4158ab0 0x7f60b41f3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34720/0 (socket says 172.21.15.97:34720) 2024-03-29T04:08:14.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60ba89d640 1 -- 172.21.15.97:0/1195278716 learned_addr learned my addr 172.21.15.97:0/1195278716 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:14.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60ba09c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60b40ba000 0x7f60b4161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60b989b640 1 --2- 172.21.15.97:0/1195278716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60b415e780 0x7f60b4158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60ba89d640 1 -- 172.21.15.97:0/1195278716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60b415e780 msgr2=0x7f60b4158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60ba89d640 1 --2- 172.21.15.97:0/1195278716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60b415e780 0x7f60b4158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60ba89d640 1 -- 172.21.15.97:0/1195278716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60b40ba000 msgr2=0x7f60b4161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:14.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60ba89d640 1 --2- 172.21.15.97:0/1195278716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60b40ba000 0x7f60b4161390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:14.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60ba89d640 1 -- 172.21.15.97:0/1195278716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b4157e00 con 0x7f60b4158ab0 2024-03-29T04:08:14.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60ba89d640 1 --2- 172.21.15.97:0/1195278716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60b4158ab0 0x7f60b41f3470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00caf0 tx=0x7f60ac0020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:14.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60aaffd640 1 -- 172.21.15.97:0/1195278716 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac018070 con 0x7f60b4158ab0 2024-03-29T04:08:14.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60aaffd640 1 -- 172.21.15.97:0/1195278716 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac00a760 con 0x7f60b4158ab0 2024-03-29T04:08:14.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60bc300640 1 -- 172.21.15.97:0/1195278716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b41f4280 con 0x7f60b4158ab0 2024-03-29T04:08:14.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60aaffd640 1 -- 172.21.15.97:0/1195278716 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00ab50 con 0x7f60b4158ab0 2024-03-29T04:08:14.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.991+0000 7f60bc300640 1 -- 172.21.15.97:0/1195278716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b41f4840 con 0x7f60b4158ab0 2024-03-29T04:08:14.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.993+0000 7f60bc300640 1 -- 172.21.15.97:0/1195278716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b4068640 con 0x7f60b4158ab0 2024-03-29T04:08:14.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.993+0000 7f60aaffd640 1 -- 172.21.15.97:0/1195278716 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f60ac01b070 con 0x7f60b4158ab0 2024-03-29T04:08:14.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.993+0000 7f60aaffd640 1 --2- 172.21.15.97:0/1195278716 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f609806c530 0x7f609806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:14.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.993+0000 7f60ba09c640 1 --2- 172.21.15.97:0/1195278716 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f609806c530 0x7f609806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:14.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.993+0000 7f60aaffd640 1 -- 172.21.15.97:0/1195278716 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f60ac08ae50 con 0x7f60b4158ab0 2024-03-29T04:08:14.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.994+0000 7f60ba09c640 1 --2- 172.21.15.97:0/1195278716 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f609806c530 0x7f609806e9f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f60b0034000 tx=0x7f60b002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:14.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:14.998+0000 7f60aaffd640 1 -- 172.21.15.97:0/1195278716 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f60ac058e20 con 0x7f60b4158ab0 2024-03-29T04:08:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.200+0000 7f60bc300640 1 -- 172.21.15.97:0/1195278716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f60b40d1ac0 con 0x7f60b4158ab0 2024-03-29T04:08:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.201+0000 7f60aaffd640 1 -- 172.21.15.97:0/1195278716 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f60ac05d120 con 0x7f60b4158ab0 2024-03-29T04:08:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-03-29T04:08:15.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.203+0000 7f60bc300640 1 -- 172.21.15.97:0/1195278716 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f609806c530 msgr2=0x7f609806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:15.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.203+0000 7f60bc300640 1 --2- 172.21.15.97:0/1195278716 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f609806c530 0x7f609806e9f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f60b0034000 tx=0x7f60b002f000 comp rx=0 tx=0).stop 2024-03-29T04:08:15.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.203+0000 7f60bc300640 1 -- 172.21.15.97:0/1195278716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60b4158ab0 msgr2=0x7f60b41f3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:15.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.203+0000 7f60bc300640 1 --2- 172.21.15.97:0/1195278716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60b4158ab0 0x7f60b41f3470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00caf0 tx=0x7f60ac0020f0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.203+0000 7f60bc300640 1 -- 172.21.15.97:0/1195278716 shutdown_connections 2024-03-29T04:08:15.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.203+0000 7f60bc300640 1 --2- 172.21.15.97:0/1195278716 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f609806c530 0x7f609806e9f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.203+0000 7f60bc300640 1 --2- 172.21.15.97:0/1195278716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60b4158ab0 0x7f60b41f3470 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.203+0000 7f60bc300640 1 --2- 172.21.15.97:0/1195278716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f60b415e780 0x7f60b4158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.203+0000 7f60bc300640 1 --2- 172.21.15.97:0/1195278716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60b40ba000 0x7f60b4161390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.203+0000 7f60bc300640 1 -- 172.21.15.97:0/1195278716 >> 172.21.15.97:0/1195278716 conn(0x7f60b408bfc0 msgr2=0x7f60b406ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:15.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.203+0000 7f60bc300640 1 -- 172.21.15.97:0/1195278716 shutdown_connections 2024-03-29T04:08:15.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.203+0000 7f60bc300640 1 -- 172.21.15.97:0/1195278716 wait complete. 2024-03-29T04:08:15.193 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-29T04:08:15.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:15.194 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-29T04:08:15.221 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T04:08:15.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:15.222 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T04:08:15.276 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:08:15.276 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:15.293 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:08:15.293 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:15.348 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:08:15.349 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:15.404 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:15.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.527+0000 7f3b7e5b5640 1 Processor -- start 2024-03-29T04:08:15.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.527+0000 7f3b7e5b5640 1 -- start start 2024-03-29T04:08:15.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b7e5b5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3b780b3140 0x7f3b7806c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:15.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b7e5b5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b7806cb60 0x7f3b781626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:15.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b7e5b5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b7806cf90 con 0x7f3b780b2a30 2024-03-29T04:08:15.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b7e5b5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f3b780d0ff0 con 0x7f3b7806cb60 2024-03-29T04:08:15.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b7e5b5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f3b78160270 con 0x7f3b780b3140 2024-03-29T04:08:15.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b7cb52640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b7806cb60 0x7f3b781626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:15.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b77fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3b780b2a30 0x7f3b780d1480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35438/0 (socket says 172.21.15.97:35438) 2024-03-29T04:08:15.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b77fff640 1 -- 172.21.15.97:0/2598548433 learned_addr learned my addr 172.21.15.97:0/2598548433 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:15.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b777fe640 1 --2- 172.21.15.97:0/2598548433 >> v2:172.21.15.154:3300/0 conn(0x7f3b780b3140 0x7f3b7806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:15.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b7cb52640 1 -- 172.21.15.97:0/2598548433 >> v2:172.21.15.154:3300/0 conn(0x7f3b780b3140 msgr2=0x7f3b7806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:15.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b7cb52640 1 --2- 172.21.15.97:0/2598548433 >> v2:172.21.15.154:3300/0 conn(0x7f3b780b3140 0x7f3b7806c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b7cb52640 1 -- 172.21.15.97:0/2598548433 >> v1:172.21.15.97:6789/0 conn(0x7f3b780b2a30 legacy=0x7f3b780d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:15.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b7cb52640 1 -- 172.21.15.97:0/2598548433 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b780b2e10 con 0x7f3b7806cb60 2024-03-29T04:08:15.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b76ffd640 1 -- 172.21.15.97:0/2598548433 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 178497630 0 0) 0x7f3b7806cf90 con 0x7f3b780b2a30 2024-03-29T04:08:15.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.528+0000 7f3b777fe640 1 --2- 172.21.15.97:0/2598548433 >> v2:172.21.15.154:3300/0 conn(0x7f3b780b3140 0x7f3b7806c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:15.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.529+0000 7f3b7cb52640 1 --2- 172.21.15.97:0/2598548433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7806cb60 0x7f3b781626c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3b7806fa60 tx=0x7f3b7002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef9b353daf38c5c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:15.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.529+0000 7f3b76ffd640 1 -- 172.21.15.97:0/2598548433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70039070 con 0x7f3b7806cb60 2024-03-29T04:08:15.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.529+0000 7f3b76ffd640 1 -- 172.21.15.97:0/2598548433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7002cc80 con 0x7f3b7806cb60 2024-03-29T04:08:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.529+0000 7f3b76ffd640 1 -- 172.21.15.97:0/2598548433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70034660 con 0x7f3b7806cb60 2024-03-29T04:08:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.529+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2598548433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7806cb60 msgr2=0x7f3b781626c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.529+0000 7f3b7e5b5640 1 --2- 172.21.15.97:0/2598548433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7806cb60 0x7f3b781626c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3b7806fa60 tx=0x7f3b7002bed0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.530+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2598548433 shutdown_connections 2024-03-29T04:08:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.530+0000 7f3b7e5b5640 1 --2- 172.21.15.97:0/2598548433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b7806cb60 0x7f3b781626c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.530+0000 7f3b7e5b5640 1 --2- 172.21.15.97:0/2598548433 >> v2:172.21.15.154:3300/0 conn(0x7f3b780b3140 0x7f3b7806c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.530+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2598548433 >> 172.21.15.97:0/2598548433 conn(0x7f3b780b3fc0 msgr2=0x7f3b780b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.530+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2598548433 shutdown_connections 2024-03-29T04:08:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.530+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2598548433 wait complete. 2024-03-29T04:08:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.531+0000 7f3b7e5b5640 1 Processor -- start 2024-03-29T04:08:15.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.531+0000 7f3b7e5b5640 1 -- start start 2024-03-29T04:08:15.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7e5b5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b7806cb60 0x7f3b780cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:15.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7e5b5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b780b2a30 0x7f3b780c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7e5b5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b780b3140 0x7f3b780c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7e5b5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3b780d0c80 con 0x7f3b780b3140 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7e5b5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3b780d09a0 con 0x7f3b780b2a30 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7e5b5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3b780d1b90 con 0x7f3b7806cb60 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7cb52640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b780b3140 0x7f3b780c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7cb52640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b780b3140 0x7f3b780c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34726/0 (socket says 172.21.15.97:34726) 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7cb52640 1 -- 172.21.15.97:0/2958994540 learned_addr learned my addr 172.21.15.97:0/2958994540 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b77fff640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b7806cb60 0x7f3b780cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b777fe640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b780b2a30 0x7f3b780c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7cb52640 1 -- 172.21.15.97:0/2958994540 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b7806cb60 msgr2=0x7f3b780cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7cb52640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b7806cb60 0x7f3b780cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7cb52640 1 -- 172.21.15.97:0/2958994540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b780b2a30 msgr2=0x7f3b780c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7cb52640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b780b2a30 0x7f3b780c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7cb52640 1 -- 172.21.15.97:0/2958994540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b780c5f30 con 0x7f3b780b3140 2024-03-29T04:08:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7cb52640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b780b3140 0x7f3b780c9d50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009e80 tx=0x7f3b70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b74ff9640 1 -- 172.21.15.97:0/2958994540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70039050 con 0x7f3b780b3140 2024-03-29T04:08:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b74ff9640 1 -- 172.21.15.97:0/2958994540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b70045070 con 0x7f3b780b3140 2024-03-29T04:08:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2958994540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b780c6210 con 0x7f3b780b3140 2024-03-29T04:08:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b74ff9640 1 -- 172.21.15.97:0/2958994540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7003e830 con 0x7f3b780b3140 2024-03-29T04:08:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.532+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2958994540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b780c67d0 con 0x7f3b780b3140 2024-03-29T04:08:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.533+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2958994540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b78068d50 con 0x7f3b780b3140 2024-03-29T04:08:15.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.534+0000 7f3b74ff9640 1 -- 172.21.15.97:0/2958994540 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3b7003e9d0 con 0x7f3b780b3140 2024-03-29T04:08:15.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.534+0000 7f3b74ff9640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3b6406c530 0x7f3b6406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:15.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.535+0000 7f3b77fff640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3b6406c530 0x7f3b6406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:15.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.535+0000 7f3b74ff9640 1 -- 172.21.15.97:0/2958994540 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3b700ae2d0 con 0x7f3b780b3140 2024-03-29T04:08:15.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.535+0000 7f3b77fff640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3b6406c530 0x7f3b6406e9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3b68000f60 tx=0x7f3b68007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:15.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.540+0000 7f3b74ff9640 1 -- 172.21.15.97:0/2958994540 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3b7007c330 con 0x7f3b780b3140 2024-03-29T04:08:15.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.714+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2958994540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b780b4bf0 con 0x7f3b780b3140 2024-03-29T04:08:15.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.715+0000 7f3b74ff9640 1 -- 172.21.15.97:0/2958994540 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f3b70080630 con 0x7f3b780b3140 2024-03-29T04:08:15.695 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:15.695 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:15.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.717+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2958994540 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3b6406c530 msgr2=0x7f3b6406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:15.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.717+0000 7f3b7e5b5640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3b6406c530 0x7f3b6406e9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3b68000f60 tx=0x7f3b68007d50 comp rx=0 tx=0).stop 2024-03-29T04:08:15.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.717+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2958994540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b780b3140 msgr2=0x7f3b780c9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:15.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.717+0000 7f3b7e5b5640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b780b3140 0x7f3b780c9d50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009e80 tx=0x7f3b70004a40 comp rx=0 tx=0).stop 2024-03-29T04:08:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.717+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2958994540 shutdown_connections 2024-03-29T04:08:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.717+0000 7f3b7e5b5640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3b6406c530 0x7f3b6406e9f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.717+0000 7f3b7e5b5640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b780b3140 0x7f3b780c9d50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.717+0000 7f3b7e5b5640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b780b2a30 0x7f3b780c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.717+0000 7f3b7e5b5640 1 --2- 172.21.15.97:0/2958994540 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b7806cb60 0x7f3b780cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.717+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2958994540 >> 172.21.15.97:0/2958994540 conn(0x7f3b780b3fc0 msgr2=0x7f3b780747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.718+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2958994540 shutdown_connections 2024-03-29T04:08:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.718+0000 7f3b7e5b5640 1 -- 172.21.15.97:0/2958994540 wait complete. 2024-03-29T04:08:15.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:07:38.618704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:07:39.724339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:08:15.709 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:08:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.833+0000 7fa266f74640 1 Processor -- start 2024-03-29T04:08:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.833+0000 7fa266f74640 1 -- start start 2024-03-29T04:08:15.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa266f74640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa26015b4f0 0x7fa2600c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:15.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa266f74640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2600b2100 0x7fa2600b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa266f74640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa26015bd30 con 0x7fa26015b8d0 2024-03-29T04:08:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa266f74640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa26015bfa0 con 0x7fa26015b4f0 2024-03-29T04:08:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa266f74640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa26015c280 con 0x7fa2600b2100 2024-03-29T04:08:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa264d10640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa26015b4f0 0x7fa2600c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa264d10640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa26015b4f0 0x7fa2600c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34728/0 (socket says 172.21.15.97:34728) 2024-03-29T04:08:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa264d10640 1 -- 172.21.15.97:0/3471936577 learned_addr learned my addr 172.21.15.97:0/3471936577 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa265511640 1 --2- 172.21.15.97:0/3471936577 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2600b2100 0x7fa2600b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa264d10640 1 -- 172.21.15.97:0/3471936577 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2600b2100 msgr2=0x7fa2600b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa264d10640 1 --2- 172.21.15.97:0/3471936577 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2600b2100 0x7fa2600b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa264d10640 1 -- 172.21.15.97:0/3471936577 >> v1:172.21.15.154:6789/0 conn(0x7fa26015b8d0 legacy=0x7fa2600ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa264d10640 1 -- 172.21.15.97:0/3471936577 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26015c560 con 0x7fa26015b4f0 2024-03-29T04:08:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.834+0000 7fa265511640 1 --2- 172.21.15.97:0/3471936577 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2600b2100 0x7fa2600b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:15.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.835+0000 7fa264d10640 1 --2- 172.21.15.97:0/3471936577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa26015b4f0 0x7fa2600c6c50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa258009e50 tx=0x7fa25802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01a06e819126f12 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.835+0000 7fa25f7fe640 1 -- 172.21.15.97:0/3471936577 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25802ca70 con 0x7fa26015b4f0 2024-03-29T04:08:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.835+0000 7fa25f7fe640 1 -- 172.21.15.97:0/3471936577 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa25802cc10 con 0x7fa26015b4f0 2024-03-29T04:08:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.835+0000 7fa25f7fe640 1 -- 172.21.15.97:0/3471936577 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2580343f0 con 0x7fa26015b4f0 2024-03-29T04:08:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.836+0000 7fa266f74640 1 -- 172.21.15.97:0/3471936577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa26015b4f0 msgr2=0x7fa2600c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.836+0000 7fa266f74640 1 --2- 172.21.15.97:0/3471936577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa26015b4f0 0x7fa2600c6c50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa258009e50 tx=0x7fa25802bed0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.836+0000 7fa266f74640 1 -- 172.21.15.97:0/3471936577 shutdown_connections 2024-03-29T04:08:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.836+0000 7fa266f74640 1 --2- 172.21.15.97:0/3471936577 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2600b2100 0x7fa2600b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.836+0000 7fa266f74640 1 --2- 172.21.15.97:0/3471936577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa26015b4f0 0x7fa2600c6c50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.836+0000 7fa266f74640 1 -- 172.21.15.97:0/3471936577 >> 172.21.15.97:0/3471936577 conn(0x7fa2600d0450 msgr2=0x7fa2600d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.836+0000 7fa266f74640 1 -- 172.21.15.97:0/3471936577 shutdown_connections 2024-03-29T04:08:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.836+0000 7fa266f74640 1 -- 172.21.15.97:0/3471936577 wait complete. 2024-03-29T04:08:15.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.837+0000 7fa266f74640 1 Processor -- start 2024-03-29T04:08:15.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.837+0000 7fa266f74640 1 -- start start 2024-03-29T04:08:15.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa266f74640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2600b2100 0x7fa2600ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:15.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa266f74640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa26015b4f0 0x7fa26014b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:15.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa266f74640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa26015b8d0 0x7fa260157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa266f74640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa2600b4d50 con 0x7fa2600b2100 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa266f74640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa2600b4bd0 con 0x7fa26015b4f0 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa266f74640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa2600ce170 con 0x7fa26015b8d0 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa264d10640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2600b2100 0x7fa2600ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa264d10640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2600b2100 0x7fa2600ae150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34736/0 (socket says 172.21.15.97:34736) 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa264d10640 1 -- 172.21.15.97:0/1225624671 learned_addr learned my addr 172.21.15.97:0/1225624671 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa25ffff640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa26015b4f0 0x7fa26014b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa265511640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa26015b8d0 0x7fa260157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa264d10640 1 -- 172.21.15.97:0/1225624671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa26015b8d0 msgr2=0x7fa260157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa264d10640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa26015b8d0 0x7fa260157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa264d10640 1 -- 172.21.15.97:0/1225624671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa26015b4f0 msgr2=0x7fa26014b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa264d10640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa26015b4f0 0x7fa26014b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa264d10640 1 -- 172.21.15.97:0/1225624671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa260157860 con 0x7fa2600b2100 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa265511640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa26015b8d0 0x7fa260157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa264d10640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2600b2100 0x7fa2600ae150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa258000f80 tx=0x7fa258004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa25d7fa640 1 -- 172.21.15.97:0/1225624671 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258043020 con 0x7fa2600b2100 2024-03-29T04:08:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa25d7fa640 1 -- 172.21.15.97:0/1225624671 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa25803e450 con 0x7fa2600b2100 2024-03-29T04:08:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa266f74640 1 -- 172.21.15.97:0/1225624671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa260157b00 con 0x7fa2600b2100 2024-03-29T04:08:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa25d7fa640 1 -- 172.21.15.97:0/1225624671 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258009510 con 0x7fa2600b2100 2024-03-29T04:08:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.838+0000 7fa266f74640 1 -- 172.21.15.97:0/1225624671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2601580c0 con 0x7fa2600b2100 2024-03-29T04:08:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.839+0000 7fa266f74640 1 -- 172.21.15.97:0/1225624671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa26009e520 con 0x7fa2600b2100 2024-03-29T04:08:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.840+0000 7fa25d7fa640 1 -- 172.21.15.97:0/1225624671 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa258046020 con 0x7fa2600b2100 2024-03-29T04:08:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.840+0000 7fa25d7fa640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa24c06c530 0x7fa24c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.840+0000 7fa25d7fa640 1 -- 172.21.15.97:0/1225624671 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa2580adea0 con 0x7fa2600b2100 2024-03-29T04:08:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.840+0000 7fa25ffff640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa24c06c530 0x7fa24c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.841+0000 7fa25ffff640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa24c06c530 0x7fa24c06e9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa26006d4f0 tx=0x7fa254009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:15.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:15.844+0000 7fa25d7fa640 1 -- 172.21.15.97:0/1225624671 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa25804d030 con 0x7fa2600b2100 2024-03-29T04:08:15.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.017+0000 7fa266f74640 1 -- 172.21.15.97:0/1225624671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa26004f9d0 con 0x7fa2600b2100 2024-03-29T04:08:15.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.018+0000 7fa25d7fa640 1 -- 172.21.15.97:0/1225624671 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa25807c4c0 con 0x7fa2600b2100 2024-03-29T04:08:15.997 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:08:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.020+0000 7fa266f74640 1 -- 172.21.15.97:0/1225624671 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa24c06c530 msgr2=0x7fa24c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.020+0000 7fa266f74640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa24c06c530 0x7fa24c06e9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa26006d4f0 tx=0x7fa254009040 comp rx=0 tx=0).stop 2024-03-29T04:08:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.020+0000 7fa266f74640 1 -- 172.21.15.97:0/1225624671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2600b2100 msgr2=0x7fa2600ae150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.020+0000 7fa266f74640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2600b2100 0x7fa2600ae150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa258000f80 tx=0x7fa258004a40 comp rx=0 tx=0).stop 2024-03-29T04:08:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.020+0000 7fa266f74640 1 -- 172.21.15.97:0/1225624671 shutdown_connections 2024-03-29T04:08:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.020+0000 7fa266f74640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa24c06c530 0x7fa24c06e9f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.020+0000 7fa266f74640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa26015b8d0 0x7fa260157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.020+0000 7fa266f74640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa26015b4f0 0x7fa26014b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.020+0000 7fa266f74640 1 --2- 172.21.15.97:0/1225624671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2600b2100 0x7fa2600ae150 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.020+0000 7fa266f74640 1 -- 172.21.15.97:0/1225624671 >> 172.21.15.97:0/1225624671 conn(0x7fa2600d0450 msgr2=0x7fa2600ad1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:16.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.020+0000 7fa266f74640 1 -- 172.21.15.97:0/1225624671 shutdown_connections 2024-03-29T04:08:16.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.020+0000 7fa266f74640 1 -- 172.21.15.97:0/1225624671 wait complete. 2024-03-29T04:08:16.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:08:16.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.133+0000 7ff94ab3d640 1 Processor -- start 2024-03-29T04:08:16.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.133+0000 7ff94ab3d640 1 -- start start 2024-03-29T04:08:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.134+0000 7ff94ab3d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff94415e920 0x7ff9440d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.134+0000 7ff94ab3d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9440d3780 con 0x7ff9440cfde0 2024-03-29T04:08:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.134+0000 7ff94ab3d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff94415ed00 con 0x7ff9440d16a0 2024-03-29T04:08:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.134+0000 7ff94ab3d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7ff9440d21e0 con 0x7ff94415e920 2024-03-29T04:08:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.134+0000 7ff9488d9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff94415e920 0x7ff9440d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.134+0000 7ff943fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff9440cfde0 0x7ff9440d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35444/0 (socket says 172.21.15.97:35444) 2024-03-29T04:08:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.134+0000 7ff943fff640 1 -- 172.21.15.97:0/166392400 learned_addr learned my addr 172.21.15.97:0/166392400 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.135+0000 7ff9488d9640 1 -- 172.21.15.97:0/166392400 >> v1:172.21.15.154:6789/0 conn(0x7ff9440d16a0 legacy=0x7ff94406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.135+0000 7ff9488d9640 1 -- 172.21.15.97:0/166392400 >> v1:172.21.15.97:6789/0 conn(0x7ff9440cfde0 legacy=0x7ff9440d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.135+0000 7ff9437fe640 1 -- 172.21.15.97:0/166392400 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 48533712 0 0) 0x7ff938001110 con 0x7ff9440cfde0 2024-03-29T04:08:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.135+0000 7ff9488d9640 1 -- 172.21.15.97:0/166392400 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9440d2480 con 0x7ff94415e920 2024-03-29T04:08:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.135+0000 7ff9488d9640 1 --2- 172.21.15.97:0/166392400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff94415e920 0x7ff9440d3240 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff9340097a0 tx=0x7ff93402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e15f801b0448028 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.135+0000 7ff9437fe640 1 -- 172.21.15.97:0/166392400 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934034070 con 0x7ff94415e920 2024-03-29T04:08:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.135+0000 7ff9437fe640 1 -- 172.21.15.97:0/166392400 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff934030610 con 0x7ff94415e920 2024-03-29T04:08:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.135+0000 7ff9437fe640 1 -- 172.21.15.97:0/166392400 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9340309e0 con 0x7ff94415e920 2024-03-29T04:08:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.136+0000 7ff94ab3d640 1 -- 172.21.15.97:0/166392400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff94415e920 msgr2=0x7ff9440d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.136+0000 7ff94ab3d640 1 --2- 172.21.15.97:0/166392400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff94415e920 0x7ff9440d3240 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff9340097a0 tx=0x7ff93402b960 comp rx=0 tx=0).stop 2024-03-29T04:08:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.136+0000 7ff94ab3d640 1 -- 172.21.15.97:0/166392400 shutdown_connections 2024-03-29T04:08:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.136+0000 7ff94ab3d640 1 --2- 172.21.15.97:0/166392400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff94415e920 0x7ff9440d3240 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.136+0000 7ff94ab3d640 1 -- 172.21.15.97:0/166392400 >> 172.21.15.97:0/166392400 conn(0x7ff94408bfc0 msgr2=0x7ff9440d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.136+0000 7ff94ab3d640 1 -- 172.21.15.97:0/166392400 shutdown_connections 2024-03-29T04:08:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.136+0000 7ff94ab3d640 1 -- 172.21.15.97:0/166392400 wait complete. 2024-03-29T04:08:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.137+0000 7ff94ab3d640 1 Processor -- start 2024-03-29T04:08:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.137+0000 7ff94ab3d640 1 -- start start 2024-03-29T04:08:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff94ab3d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9440cfde0 0x7ff944152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff94ab3d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9440d16a0 0x7ff9440b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff94ab3d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff94415e920 0x7ff9440b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff94ab3d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff9440d2d20 con 0x7ff9440cfde0 2024-03-29T04:08:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff94ab3d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff9440d2a40 con 0x7ff9440d16a0 2024-03-29T04:08:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff94ab3d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff9440d4ef0 con 0x7ff94415e920 2024-03-29T04:08:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff9488d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9440cfde0 0x7ff944152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff9488d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9440cfde0 0x7ff944152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34764/0 (socket says 172.21.15.97:34764) 2024-03-29T04:08:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff9488d9640 1 -- 172.21.15.97:0/3128350299 learned_addr learned my addr 172.21.15.97:0/3128350299 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff9490da640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff94415e920 0x7ff9440b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff943fff640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9440d16a0 0x7ff9440b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff9488d9640 1 -- 172.21.15.97:0/3128350299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff94415e920 msgr2=0x7ff9440b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff9488d9640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff94415e920 0x7ff9440b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff9488d9640 1 -- 172.21.15.97:0/3128350299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9440d16a0 msgr2=0x7ff9440b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff9488d9640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9440d16a0 0x7ff9440b6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.138+0000 7ff9488d9640 1 -- 172.21.15.97:0/3128350299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9440b45b0 con 0x7ff9440cfde0 2024-03-29T04:08:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.139+0000 7ff9488d9640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9440cfde0 0x7ff944152f50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff934000c00 tx=0x7ff9340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.139+0000 7ff9417fa640 1 -- 172.21.15.97:0/3128350299 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934034040 con 0x7ff9440cfde0 2024-03-29T04:08:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.139+0000 7ff9417fa640 1 -- 172.21.15.97:0/3128350299 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff934004030 con 0x7ff9440cfde0 2024-03-29T04:08:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.139+0000 7ff94ab3d640 1 -- 172.21.15.97:0/3128350299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9441f5020 con 0x7ff9440cfde0 2024-03-29T04:08:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.139+0000 7ff9417fa640 1 -- 172.21.15.97:0/3128350299 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934004420 con 0x7ff9440cfde0 2024-03-29T04:08:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.139+0000 7ff94ab3d640 1 -- 172.21.15.97:0/3128350299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9441f55a0 con 0x7ff9440cfde0 2024-03-29T04:08:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.140+0000 7ff94ab3d640 1 -- 172.21.15.97:0/3128350299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9440a2560 con 0x7ff9440cfde0 2024-03-29T04:08:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.140+0000 7ff9417fa640 1 -- 172.21.15.97:0/3128350299 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff93404c060 con 0x7ff9440cfde0 2024-03-29T04:08:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.140+0000 7ff9417fa640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff93006c530 0x7ff93006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.141+0000 7ff9417fa640 1 -- 172.21.15.97:0/3128350299 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff9340ad1f0 con 0x7ff9440cfde0 2024-03-29T04:08:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.141+0000 7ff943fff640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff93006c530 0x7ff93006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.141+0000 7ff943fff640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff93006c530 0x7ff93006e9f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff9441f6060 tx=0x7ff938007e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:16.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.145+0000 7ff9417fa640 1 -- 172.21.15.97:0/3128350299 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff93407b2e0 con 0x7ff9440cfde0 2024-03-29T04:08:16.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.318+0000 7ff94ab3d640 1 -- 172.21.15.97:0/3128350299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9440c7ee0 con 0x7ff9440cfde0 2024-03-29T04:08:16.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.318+0000 7ff9417fa640 1 -- 172.21.15.97:0/3128350299 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff93407f5e0 con 0x7ff9440cfde0 2024-03-29T04:08:16.298 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:08:16.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.320+0000 7ff94ab3d640 1 -- 172.21.15.97:0/3128350299 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff93006c530 msgr2=0x7ff93006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.320+0000 7ff94ab3d640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff93006c530 0x7ff93006e9f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff9441f6060 tx=0x7ff938007e40 comp rx=0 tx=0).stop 2024-03-29T04:08:16.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.320+0000 7ff94ab3d640 1 -- 172.21.15.97:0/3128350299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9440cfde0 msgr2=0x7ff944152f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.320+0000 7ff94ab3d640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9440cfde0 0x7ff944152f50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff934000c00 tx=0x7ff9340046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.320+0000 7ff94ab3d640 1 -- 172.21.15.97:0/3128350299 shutdown_connections 2024-03-29T04:08:16.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.320+0000 7ff94ab3d640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff93006c530 0x7ff93006e9f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.320+0000 7ff94ab3d640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff94415e920 0x7ff9440b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.320+0000 7ff94ab3d640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9440d16a0 0x7ff9440b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.320+0000 7ff94ab3d640 1 --2- 172.21.15.97:0/3128350299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9440cfde0 0x7ff944152f50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.320+0000 7ff94ab3d640 1 -- 172.21.15.97:0/3128350299 >> 172.21.15.97:0/3128350299 conn(0x7ff94408bfc0 msgr2=0x7ff94415dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:16.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.320+0000 7ff94ab3d640 1 -- 172.21.15.97:0/3128350299 shutdown_connections 2024-03-29T04:08:16.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.320+0000 7ff94ab3d640 1 -- 172.21.15.97:0/3128350299 wait complete. 2024-03-29T04:08:16.310 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:08:16.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.433+0000 7fb84ea7e640 1 Processor -- start 2024-03-29T04:08:16.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.433+0000 7fb84ea7e640 1 -- start start 2024-03-29T04:08:16.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.434+0000 7fb84ea7e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb8480c6c50 0x7fb84815b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.434+0000 7fb84ea7e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb84815bfa0 con 0x7fb84815b4f0 2024-03-29T04:08:16.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.434+0000 7fb84ea7e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb84815c280 con 0x7fb8480b2100 2024-03-29T04:08:16.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.434+0000 7fb84ea7e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fb8480c70b0 con 0x7fb8480c6c50 2024-03-29T04:08:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.434+0000 7fb84c81a640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb84815b4f0 0x7fb8480ca950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35448/0 (socket says 172.21.15.97:35448) 2024-03-29T04:08:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.434+0000 7fb84c81a640 1 -- 172.21.15.97:0/1713272146 learned_addr learned my addr 172.21.15.97:0/1713272146 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.434+0000 7fb847fff640 1 --2- 172.21.15.97:0/1713272146 >> v2:172.21.15.154:3300/0 conn(0x7fb8480c6c50 0x7fb84815b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.434+0000 7fb8477fe640 1 -- 172.21.15.97:0/1713272146 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3038672638 0 0) 0x7fb84815bfa0 con 0x7fb84815b4f0 2024-03-29T04:08:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.434+0000 7fb8477fe640 1 -- 172.21.15.97:0/1713272146 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb830003520 con 0x7fb84815b4f0 2024-03-29T04:08:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.434+0000 7fb847fff640 1 -- 172.21.15.97:0/1713272146 >> v1:172.21.15.154:6789/0 conn(0x7fb8480b2100 legacy=0x7fb8480ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.434+0000 7fb8477fe640 1 -- 172.21.15.97:0/1713272146 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1125395417 0 0) 0x7fb84815c280 con 0x7fb8480b2100 2024-03-29T04:08:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.434+0000 7fb847fff640 1 -- 172.21.15.97:0/1713272146 >> v1:172.21.15.97:6789/0 conn(0x7fb84815b4f0 legacy=0x7fb8480ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.434+0000 7fb847fff640 1 -- 172.21.15.97:0/1713272146 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84815c560 con 0x7fb8480c6c50 2024-03-29T04:08:16.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.435+0000 7fb847fff640 1 --2- 172.21.15.97:0/1713272146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8480c6c50 0x7fb84815b8d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0097a0 tx=0x7fb83c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8d89a3ac58c9772 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.435+0000 7fb8477fe640 1 -- 172.21.15.97:0/1713272146 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c035020 con 0x7fb8480c6c50 2024-03-29T04:08:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.435+0000 7fb8477fe640 1 -- 172.21.15.97:0/1713272146 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb83c031830 con 0x7fb8480c6c50 2024-03-29T04:08:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.435+0000 7fb8477fe640 1 -- 172.21.15.97:0/1713272146 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c031be0 con 0x7fb8480c6c50 2024-03-29T04:08:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.436+0000 7fb84ea7e640 1 -- 172.21.15.97:0/1713272146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8480c6c50 msgr2=0x7fb84815b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.436+0000 7fb84ea7e640 1 --2- 172.21.15.97:0/1713272146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8480c6c50 0x7fb84815b8d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0097a0 tx=0x7fb83c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.436+0000 7fb84ea7e640 1 -- 172.21.15.97:0/1713272146 shutdown_connections 2024-03-29T04:08:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.436+0000 7fb84ea7e640 1 --2- 172.21.15.97:0/1713272146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8480c6c50 0x7fb84815b8d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.436+0000 7fb84ea7e640 1 -- 172.21.15.97:0/1713272146 >> 172.21.15.97:0/1713272146 conn(0x7fb8480d0450 msgr2=0x7fb8480d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.436+0000 7fb84ea7e640 1 -- 172.21.15.97:0/1713272146 shutdown_connections 2024-03-29T04:08:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.436+0000 7fb84ea7e640 1 -- 172.21.15.97:0/1713272146 wait complete. 2024-03-29T04:08:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.437+0000 7fb84ea7e640 1 Processor -- start 2024-03-29T04:08:16.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.437+0000 7fb84ea7e640 1 -- start start 2024-03-29T04:08:16.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84ea7e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8480b2100 0x7fb84815dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84ea7e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8480c6c50 0x7fb84806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84ea7e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb84815b4f0 0x7fb84806d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84ea7e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb8480b4d50 con 0x7fb84815b4f0 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84ea7e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb8480b4bd0 con 0x7fb8480c6c50 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84ea7e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb8480ce880 con 0x7fb8480b2100 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84d01b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb84815b4f0 0x7fb84806d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84d01b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb84815b4f0 0x7fb84806d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34774/0 (socket says 172.21.15.97:34774) 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84d01b640 1 -- 172.21.15.97:0/660633610 learned_addr learned my addr 172.21.15.97:0/660633610 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84c81a640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8480b2100 0x7fb84815dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb847fff640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8480c6c50 0x7fb84806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84d01b640 1 -- 172.21.15.97:0/660633610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8480b2100 msgr2=0x7fb84815dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84d01b640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8480b2100 0x7fb84815dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84d01b640 1 -- 172.21.15.97:0/660633610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8480c6c50 msgr2=0x7fb84806f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84d01b640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8480c6c50 0x7fb84806f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb84d01b640 1 -- 172.21.15.97:0/660633610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84815e300 con 0x7fb84815b4f0 2024-03-29T04:08:16.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.438+0000 7fb847fff640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8480c6c50 0x7fb84806f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:16.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.439+0000 7fb84d01b640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb84815b4f0 0x7fb84806d7d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb84000c760 tx=0x7fb84000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:16.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.439+0000 7fb8457fa640 1 -- 172.21.15.97:0/660633610 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840003da0 con 0x7fb84815b4f0 2024-03-29T04:08:16.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.439+0000 7fb8457fa640 1 -- 172.21.15.97:0/660633610 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb840005070 con 0x7fb84815b4f0 2024-03-29T04:08:16.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.439+0000 7fb84ea7e640 1 -- 172.21.15.97:0/660633610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84815e5e0 con 0x7fb84815b4f0 2024-03-29T04:08:16.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.439+0000 7fb8457fa640 1 -- 172.21.15.97:0/660633610 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840005460 con 0x7fb84815b4f0 2024-03-29T04:08:16.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.439+0000 7fb84ea7e640 1 -- 172.21.15.97:0/660633610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb848071970 con 0x7fb84815b4f0 2024-03-29T04:08:16.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.440+0000 7fb84ea7e640 1 -- 172.21.15.97:0/660633610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb84809e520 con 0x7fb84815b4f0 2024-03-29T04:08:16.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.440+0000 7fb8457fa640 1 -- 172.21.15.97:0/660633610 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb840005600 con 0x7fb84815b4f0 2024-03-29T04:08:16.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.440+0000 7fb8457fa640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb83406c530 0x7fb83406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.441+0000 7fb8457fa640 1 -- 172.21.15.97:0/660633610 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb84008b530 con 0x7fb84815b4f0 2024-03-29T04:08:16.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.441+0000 7fb84c81a640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb83406c530 0x7fb83406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.441+0000 7fb84c81a640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb83406c530 0x7fb83406e9f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb838005880 tx=0x7fb838009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:16.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.445+0000 7fb8457fa640 1 -- 172.21.15.97:0/660633610 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb840059500 con 0x7fb84815b4f0 2024-03-29T04:08:16.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.619+0000 7fb84ea7e640 1 -- 172.21.15.97:0/660633610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb848072a90 con 0x7fb84815b4f0 2024-03-29T04:08:16.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.619+0000 7fb8457fa640 1 -- 172.21.15.97:0/660633610 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fb84005d800 con 0x7fb84815b4f0 2024-03-29T04:08:16.599 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:08:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.621+0000 7fb84ea7e640 1 -- 172.21.15.97:0/660633610 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb83406c530 msgr2=0x7fb83406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.621+0000 7fb84ea7e640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb83406c530 0x7fb83406e9f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb838005880 tx=0x7fb838009040 comp rx=0 tx=0).stop 2024-03-29T04:08:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.621+0000 7fb84ea7e640 1 -- 172.21.15.97:0/660633610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb84815b4f0 msgr2=0x7fb84806d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.621+0000 7fb84ea7e640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb84815b4f0 0x7fb84806d7d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb84000c760 tx=0x7fb84000cc20 comp rx=0 tx=0).stop 2024-03-29T04:08:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.621+0000 7fb84ea7e640 1 -- 172.21.15.97:0/660633610 shutdown_connections 2024-03-29T04:08:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.621+0000 7fb84ea7e640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb83406c530 0x7fb83406e9f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.621+0000 7fb84ea7e640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb84815b4f0 0x7fb84806d7d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.621+0000 7fb84ea7e640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8480c6c50 0x7fb84806f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.622+0000 7fb84ea7e640 1 --2- 172.21.15.97:0/660633610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8480b2100 0x7fb84815dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.622+0000 7fb84ea7e640 1 -- 172.21.15.97:0/660633610 >> 172.21.15.97:0/660633610 conn(0x7fb8480d0450 msgr2=0x7fb8480adb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.622+0000 7fb84ea7e640 1 -- 172.21.15.97:0/660633610 shutdown_connections 2024-03-29T04:08:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.622+0000 7fb84ea7e640 1 -- 172.21.15.97:0/660633610 wait complete. 2024-03-29T04:08:16.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:08:16.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.733+0000 7fcf4efd5640 1 Processor -- start 2024-03-29T04:08:16.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.733+0000 7fcf4efd5640 1 -- start start 2024-03-29T04:08:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf4efd5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcf4815fc20 0x7fcf480c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf4efd5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf480d0f10 con 0x7fcf480ccbe0 2024-03-29T04:08:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf4efd5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf48160480 con 0x7fcf480c5020 2024-03-29T04:08:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf4efd5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fcf48161090 con 0x7fcf4815fc20 2024-03-29T04:08:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf4d572640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcf480ccbe0 0x7fcf480d0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35456/0 (socket says 172.21.15.97:35456) 2024-03-29T04:08:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf4d572640 1 -- 172.21.15.97:0/1829734364 learned_addr learned my addr 172.21.15.97:0/1829734364 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf4cd71640 1 --2- 172.21.15.97:0/1829734364 >> v2:172.21.15.154:3300/0 conn(0x7fcf4815fc20 0x7fcf480c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf477fe640 1 -- 172.21.15.97:0/1829734364 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2179641125 0 0) 0x7fcf480d0f10 con 0x7fcf480ccbe0 2024-03-29T04:08:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf477fe640 1 -- 172.21.15.97:0/1829734364 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf38003520 con 0x7fcf480ccbe0 2024-03-29T04:08:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf477fe640 1 -- 172.21.15.97:0/1829734364 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 726137487 0 0) 0x7fcf48160480 con 0x7fcf480c5020 2024-03-29T04:08:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf477fe640 1 -- 172.21.15.97:0/1829734364 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf480d0f10 con 0x7fcf480c5020 2024-03-29T04:08:16.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf4cd71640 1 -- 172.21.15.97:0/1829734364 >> v1:172.21.15.154:6789/0 conn(0x7fcf480c5020 legacy=0x7fcf480d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf4cd71640 1 -- 172.21.15.97:0/1829734364 >> v1:172.21.15.97:6789/0 conn(0x7fcf480ccbe0 legacy=0x7fcf480d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.734+0000 7fcf4cd71640 1 -- 172.21.15.97:0/1829734364 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf48160760 con 0x7fcf4815fc20 2024-03-29T04:08:16.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.735+0000 7fcf4cd71640 1 --2- 172.21.15.97:0/1829734364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf4815fc20 0x7fcf480c6140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0097a0 tx=0x7fcf3c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1277ed1ab534416 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:16.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.735+0000 7fcf477fe640 1 -- 172.21.15.97:0/1829734364 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c035020 con 0x7fcf4815fc20 2024-03-29T04:08:16.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.735+0000 7fcf477fe640 1 -- 172.21.15.97:0/1829734364 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf3c031830 con 0x7fcf4815fc20 2024-03-29T04:08:16.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.735+0000 7fcf477fe640 1 -- 172.21.15.97:0/1829734364 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c031c00 con 0x7fcf4815fc20 2024-03-29T04:08:16.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.736+0000 7fcf4efd5640 1 -- 172.21.15.97:0/1829734364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf4815fc20 msgr2=0x7fcf480c6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.736+0000 7fcf4efd5640 1 --2- 172.21.15.97:0/1829734364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf4815fc20 0x7fcf480c6140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0097a0 tx=0x7fcf3c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.736+0000 7fcf4efd5640 1 -- 172.21.15.97:0/1829734364 shutdown_connections 2024-03-29T04:08:16.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.736+0000 7fcf4efd5640 1 --2- 172.21.15.97:0/1829734364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf4815fc20 0x7fcf480c6140 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.736+0000 7fcf4efd5640 1 -- 172.21.15.97:0/1829734364 >> 172.21.15.97:0/1829734364 conn(0x7fcf4806b510 msgr2=0x7fcf4806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:16.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.736+0000 7fcf4efd5640 1 -- 172.21.15.97:0/1829734364 shutdown_connections 2024-03-29T04:08:16.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.736+0000 7fcf4efd5640 1 -- 172.21.15.97:0/1829734364 wait complete. 2024-03-29T04:08:16.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.737+0000 7fcf4efd5640 1 Processor -- start 2024-03-29T04:08:16.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.737+0000 7fcf4efd5640 1 -- start start 2024-03-29T04:08:16.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4efd5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf480c5020 0x7fcf480c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4efd5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf480ccbe0 0x7fcf480ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4efd5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4815fc20 0x7fcf4815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4efd5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481600e0 con 0x7fcf4815fc20 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4efd5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcf48160d20 con 0x7fcf480c5020 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4efd5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcf48160260 con 0x7fcf480ccbe0 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4d572640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4815fc20 0x7fcf4815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4d572640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4815fc20 0x7fcf4815e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34780/0 (socket says 172.21.15.97:34780) 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4d572640 1 -- 172.21.15.97:0/2904057072 learned_addr learned my addr 172.21.15.97:0/2904057072 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4cd71640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf480c5020 0x7fcf480c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf47fff640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf480ccbe0 0x7fcf480ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4d572640 1 -- 172.21.15.97:0/2904057072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf480ccbe0 msgr2=0x7fcf480ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4d572640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf480ccbe0 0x7fcf480ae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4d572640 1 -- 172.21.15.97:0/2904057072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf480c5020 msgr2=0x7fcf480c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4d572640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf480c5020 0x7fcf480c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4d572640 1 -- 172.21.15.97:0/2904057072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf480af5f0 con 0x7fcf4815fc20 2024-03-29T04:08:16.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4d572640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4815fc20 0x7fcf4815e2a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000cac0 tx=0x7fcf4000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:16.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf457fa640 1 -- 172.21.15.97:0/2904057072 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40017070 con 0x7fcf4815fc20 2024-03-29T04:08:16.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf457fa640 1 -- 172.21.15.97:0/2904057072 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf4000ac30 con 0x7fcf4815fc20 2024-03-29T04:08:16.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf457fa640 1 -- 172.21.15.97:0/2904057072 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40012400 con 0x7fcf4815fc20 2024-03-29T04:08:16.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.738+0000 7fcf4efd5640 1 -- 172.21.15.97:0/2904057072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf480cede0 con 0x7fcf4815fc20 2024-03-29T04:08:16.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.739+0000 7fcf4efd5640 1 -- 172.21.15.97:0/2904057072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf480cf680 con 0x7fcf4815fc20 2024-03-29T04:08:16.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.740+0000 7fcf4efd5640 1 -- 172.21.15.97:0/2904057072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf48068640 con 0x7fcf4815fc20 2024-03-29T04:08:16.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.740+0000 7fcf457fa640 1 -- 172.21.15.97:0/2904057072 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcf4001a070 con 0x7fcf4815fc20 2024-03-29T04:08:16.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.740+0000 7fcf457fa640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcf2c06c530 0x7fcf2c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:16.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.741+0000 7fcf4cd71640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcf2c06c530 0x7fcf2c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:16.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.741+0000 7fcf457fa640 1 -- 172.21.15.97:0/2904057072 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcf4008b310 con 0x7fcf4815fc20 2024-03-29T04:08:16.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.741+0000 7fcf4cd71640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcf2c06c530 0x7fcf2c06e9f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c036000 tx=0x7fcf3c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:16.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.745+0000 7fcf457fa640 1 -- 172.21.15.97:0/2904057072 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcf40059370 con 0x7fcf4815fc20 2024-03-29T04:08:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.933+0000 7fcf4efd5640 1 -- 172.21.15.97:0/2904057072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcf480cf850 con 0x7fcf4815fc20 2024-03-29T04:08:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.934+0000 7fcf457fa640 1 -- 172.21.15.97:0/2904057072 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fcf4005d670 con 0x7fcf4815fc20 2024-03-29T04:08:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-03-29T04:08:16.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.935+0000 7fcf4efd5640 1 -- 172.21.15.97:0/2904057072 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcf2c06c530 msgr2=0x7fcf2c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.936+0000 7fcf4efd5640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcf2c06c530 0x7fcf2c06e9f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c036000 tx=0x7fcf3c03d040 comp rx=0 tx=0).stop 2024-03-29T04:08:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.936+0000 7fcf4efd5640 1 -- 172.21.15.97:0/2904057072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4815fc20 msgr2=0x7fcf4815e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.936+0000 7fcf4efd5640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4815fc20 0x7fcf4815e2a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000cac0 tx=0x7fcf4000cf80 comp rx=0 tx=0).stop 2024-03-29T04:08:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.936+0000 7fcf4efd5640 1 -- 172.21.15.97:0/2904057072 shutdown_connections 2024-03-29T04:08:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.936+0000 7fcf4efd5640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcf2c06c530 0x7fcf2c06e9f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.936+0000 7fcf4efd5640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4815fc20 0x7fcf4815e2a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.936+0000 7fcf4efd5640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcf480ccbe0 0x7fcf480ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.936+0000 7fcf4efd5640 1 --2- 172.21.15.97:0/2904057072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf480c5020 0x7fcf480c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.936+0000 7fcf4efd5640 1 -- 172.21.15.97:0/2904057072 >> 172.21.15.97:0/2904057072 conn(0x7fcf4806b510 msgr2=0x7fcf4806b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.936+0000 7fcf4efd5640 1 -- 172.21.15.97:0/2904057072 shutdown_connections 2024-03-29T04:08:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:16.936+0000 7fcf4efd5640 1 -- 172.21.15.97:0/2904057072 wait complete. 2024-03-29T04:08:16.926 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-29T04:08:16.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:16.926 DEBUG:teuthology.orchestra.run.smithi154:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-29T04:08:16.954 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T04:08:16.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:16.955 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T04:08:17.010 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-29T04:08:17.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:08:17.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.130+0000 7f8897ec8640 1 Processor -- start 2024-03-29T04:08:17.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.130+0000 7f8897ec8640 1 -- start start 2024-03-29T04:08:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8897ec8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f889006c520 0x7f889006c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8897ec8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f889006e6b0 con 0x7f889006e0f0 2024-03-29T04:08:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8897ec8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f889006e990 con 0x7f889015e9a0 2024-03-29T04:08:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8897ec8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f889006f2c0 con 0x7f889006c520 2024-03-29T04:08:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8895463640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f889006e0f0 0x7f889006f750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35470/0 (socket says 172.21.15.97:35470) 2024-03-29T04:08:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8895463640 1 -- 172.21.15.97:0/2787891610 learned_addr learned my addr 172.21.15.97:0/2787891610 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8896465640 1 --2- 172.21.15.97:0/2787891610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f889006c520 0x7f889006c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8894c62640 1 -- 172.21.15.97:0/2787891610 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3845955810 0 0) 0x7f889006e6b0 con 0x7f889006e0f0 2024-03-29T04:08:17.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8894c62640 1 -- 172.21.15.97:0/2787891610 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f887c003520 con 0x7f889006e0f0 2024-03-29T04:08:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8894c62640 1 -- 172.21.15.97:0/2787891610 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3794868382 0 0) 0x7f889006e990 con 0x7f889015e9a0 2024-03-29T04:08:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8894c62640 1 -- 172.21.15.97:0/2787891610 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f889006e6b0 con 0x7f889015e9a0 2024-03-29T04:08:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8894c62640 1 -- 172.21.15.97:0/2787891610 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4263310916 0 0) 0x7f887c003520 con 0x7f889006e0f0 2024-03-29T04:08:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8896465640 1 -- 172.21.15.97:0/2787891610 >> v1:172.21.15.154:6789/0 conn(0x7f889015e9a0 legacy=0x7f8890072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.131+0000 7f8896465640 1 -- 172.21.15.97:0/2787891610 >> v1:172.21.15.97:6789/0 conn(0x7f889006e0f0 legacy=0x7f889006f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.132+0000 7f8896465640 1 -- 172.21.15.97:0/2787891610 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889006ec70 con 0x7f889006c520 2024-03-29T04:08:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.132+0000 7f8896465640 1 --2- 172.21.15.97:0/2787891610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f889006c520 0x7f889006c9b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8888009dc0 tx=0x7f888802bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e22496e617d7810 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.132+0000 7f8894c62640 1 -- 172.21.15.97:0/2787891610 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888039070 con 0x7f889006c520 2024-03-29T04:08:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.132+0000 7f8894c62640 1 -- 172.21.15.97:0/2787891610 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f888802c880 con 0x7f889006c520 2024-03-29T04:08:17.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.132+0000 7f8894c62640 1 -- 172.21.15.97:0/2787891610 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880001670 con 0x7f889006c520 2024-03-29T04:08:17.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.133+0000 7f8897ec8640 1 -- 172.21.15.97:0/2787891610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f889006c520 msgr2=0x7f889006c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.133+0000 7f8897ec8640 1 --2- 172.21.15.97:0/2787891610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f889006c520 0x7f889006c9b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8888009dc0 tx=0x7f888802bd50 comp rx=0 tx=0).stop 2024-03-29T04:08:17.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.133+0000 7f8897ec8640 1 -- 172.21.15.97:0/2787891610 shutdown_connections 2024-03-29T04:08:17.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.133+0000 7f8897ec8640 1 --2- 172.21.15.97:0/2787891610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f889006c520 0x7f889006c9b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.133+0000 7f8897ec8640 1 -- 172.21.15.97:0/2787891610 >> 172.21.15.97:0/2787891610 conn(0x7f8890072090 msgr2=0x7f88900724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:17.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.133+0000 7f8897ec8640 1 -- 172.21.15.97:0/2787891610 shutdown_connections 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.133+0000 7f8897ec8640 1 -- 172.21.15.97:0/2787891610 wait complete. 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.134+0000 7f8897ec8640 1 Processor -- start 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.134+0000 7f8897ec8640 1 -- start start 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8897ec8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f889006c520 0x7f889015ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8897ec8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f889006e0f0 0x7f88901f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8897ec8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f889015e9a0 0x7f88900af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8897ec8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88900ada80 con 0x7f889015e9a0 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8897ec8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f889006fe60 con 0x7f889006c520 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8897ec8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f88900adc00 con 0x7f889006e0f0 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8896465640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f889015e9a0 0x7f88900af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8896465640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f889015e9a0 0x7f88900af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34792/0 (socket says 172.21.15.97:34792) 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8896465640 1 -- 172.21.15.97:0/3570893148 learned_addr learned my addr 172.21.15.97:0/3570893148 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8895c64640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f889006c520 0x7f889015ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8895463640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f889006e0f0 0x7f88901f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8896465640 1 -- 172.21.15.97:0/3570893148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f889006e0f0 msgr2=0x7f88901f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8896465640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f889006e0f0 0x7f88901f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8896465640 1 -- 172.21.15.97:0/3570893148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f889006c520 msgr2=0x7f889015ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8896465640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f889006c520 0x7f889015ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8896465640 1 -- 172.21.15.97:0/3570893148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88901f7b30 con 0x7f889015e9a0 2024-03-29T04:08:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.135+0000 7f8896465640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f889015e9a0 0x7f88900af240 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8888009ee0 tx=0x7f8888004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.136+0000 7f88867fc640 1 -- 172.21.15.97:0/3570893148 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888039050 con 0x7f889015e9a0 2024-03-29T04:08:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.136+0000 7f88867fc640 1 -- 172.21.15.97:0/3570893148 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8888045070 con 0x7f889015e9a0 2024-03-29T04:08:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.136+0000 7f8897ec8640 1 -- 172.21.15.97:0/3570893148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88900af780 con 0x7f889015e9a0 2024-03-29T04:08:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.136+0000 7f88867fc640 1 -- 172.21.15.97:0/3570893148 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888009450 con 0x7f889015e9a0 2024-03-29T04:08:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.136+0000 7f8897ec8640 1 -- 172.21.15.97:0/3570893148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88900afd30 con 0x7f889015e9a0 2024-03-29T04:08:17.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.137+0000 7f8897ec8640 1 -- 172.21.15.97:0/3570893148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8890068640 con 0x7f889015e9a0 2024-03-29T04:08:17.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.137+0000 7f88867fc640 1 -- 172.21.15.97:0/3570893148 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8888004690 con 0x7f889015e9a0 2024-03-29T04:08:17.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.137+0000 7f88867fc640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f887406c530 0x7f887406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.137+0000 7f8895c64640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f887406c530 0x7f887406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.137+0000 7f88867fc640 1 -- 172.21.15.97:0/3570893148 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f88880add00 con 0x7f889015e9a0 2024-03-29T04:08:17.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.138+0000 7f8895c64640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f887406c530 0x7f887406e9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f888c0011e0 tx=0x7f888c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:17.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.142+0000 7f88867fc640 1 -- 172.21.15.97:0/3570893148 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f888807bd60 con 0x7f889015e9a0 2024-03-29T04:08:17.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.352+0000 7f8897ec8640 1 -- 172.21.15.97:0/3570893148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f889006c300 con 0x7f889015e9a0 2024-03-29T04:08:17.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.353+0000 7f88867fc640 1 -- 172.21.15.97:0/3570893148 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7f8888080060 con 0x7f889015e9a0 2024-03-29T04:08:17.333 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:17.333 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1127043830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127043830},{"type":"v1","addr":"172.21.15.154:6834","nonce":1127043830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3767188750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3767188750},{"type":"v1","addr":"172.21.15.97:6835","nonce":3767188750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3651340478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3651340478},{"type":"v1","addr":"172.21.15.97:6837","nonce":3651340478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:07:40.761466+0000","modified":"2024-03-29T04:07:42.771721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3669418265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3669418265},{"type":"v1","addr":"172.21.15.154:6835","nonce":3669418265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-29T04:08:17.334 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-03-29T04:08:17.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.355+0000 7f8897ec8640 1 -- 172.21.15.97:0/3570893148 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f887406c530 msgr2=0x7f887406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.355+0000 7f8897ec8640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f887406c530 0x7f887406e9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f888c0011e0 tx=0x7f888c009040 comp rx=0 tx=0).stop 2024-03-29T04:08:17.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.355+0000 7f8897ec8640 1 -- 172.21.15.97:0/3570893148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f889015e9a0 msgr2=0x7f88900af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.355+0000 7f8897ec8640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f889015e9a0 0x7f88900af240 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8888009ee0 tx=0x7f8888004a40 comp rx=0 tx=0).stop 2024-03-29T04:08:17.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.355+0000 7f8897ec8640 1 -- 172.21.15.97:0/3570893148 shutdown_connections 2024-03-29T04:08:17.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.355+0000 7f8897ec8640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f887406c530 0x7f887406e9f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.355+0000 7f8897ec8640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f889015e9a0 0x7f88900af240 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.355+0000 7f8897ec8640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f889006e0f0 0x7f88901f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.355+0000 7f8897ec8640 1 --2- 172.21.15.97:0/3570893148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f889006c520 0x7f889015ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.355+0000 7f8897ec8640 1 -- 172.21.15.97:0/3570893148 >> 172.21.15.97:0/3570893148 conn(0x7f8890072090 msgr2=0x7f8890160d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:17.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.355+0000 7f8897ec8640 1 -- 172.21.15.97:0/3570893148 shutdown_connections 2024-03-29T04:08:17.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.355+0000 7f8897ec8640 1 -- 172.21.15.97:0/3570893148 wait complete. 2024-03-29T04:08:17.346 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:17.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.467+0000 7fadbed72640 1 Processor -- start 2024-03-29T04:08:17.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.467+0000 7fadbed72640 1 -- start start 2024-03-29T04:08:17.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadbed72640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fadb815b370 0x7fadb80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadbed72640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fadb806f3f0 0x7fadb8072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadbed72640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb8071ae0 con 0x7fadb80730c0 2024-03-29T04:08:17.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadbed72640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fadb8071dc0 con 0x7fadb806f3f0 2024-03-29T04:08:17.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadbed72640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fadb80720a0 con 0x7fadb815b370 2024-03-29T04:08:17.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadb7fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fadb806f3f0 0x7fadb8072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadbd30f640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fadb80730c0 0x7fadb80ae020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35484/0 (socket says 172.21.15.97:35484) 2024-03-29T04:08:17.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadbd30f640 1 -- 172.21.15.97:0/2279829708 learned_addr learned my addr 172.21.15.97:0/2279829708 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:17.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadbcb0e640 1 --2- 172.21.15.97:0/2279829708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fadb815b370 0x7fadb80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadb7fff640 1 -- 172.21.15.97:0/2279829708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fadb815b370 msgr2=0x7fadb80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:17.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadb7fff640 1 --2- 172.21.15.97:0/2279829708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fadb815b370 0x7fadb80b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadb7fff640 1 -- 172.21.15.97:0/2279829708 >> v1:172.21.15.97:6789/0 conn(0x7fadb80730c0 legacy=0x7fadb80ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadb77fe640 1 -- 172.21.15.97:0/2279829708 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1901634107 0 0) 0x7fadb8071ae0 con 0x7fadb80730c0 2024-03-29T04:08:17.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.468+0000 7fadb7fff640 1 -- 172.21.15.97:0/2279829708 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb8072380 con 0x7fadb806f3f0 2024-03-29T04:08:17.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.469+0000 7fadb7fff640 1 --2- 172.21.15.97:0/2279829708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadb806f3f0 0x7fadb8072b80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fada40097a0 tx=0x7fada402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b84462d04483c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:17.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.469+0000 7fadb77fe640 1 -- 172.21.15.97:0/2279829708 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4035020 con 0x7fadb806f3f0 2024-03-29T04:08:17.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.469+0000 7fadb77fe640 1 -- 172.21.15.97:0/2279829708 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada4031830 con 0x7fadb806f3f0 2024-03-29T04:08:17.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.469+0000 7fadb77fe640 1 -- 172.21.15.97:0/2279829708 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4031c00 con 0x7fadb806f3f0 2024-03-29T04:08:17.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.469+0000 7fadbed72640 1 -- 172.21.15.97:0/2279829708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadb806f3f0 msgr2=0x7fadb8072b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.469+0000 7fadbed72640 1 --2- 172.21.15.97:0/2279829708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadb806f3f0 0x7fadb8072b80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fada40097a0 tx=0x7fada402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.469+0000 7fadbed72640 1 -- 172.21.15.97:0/2279829708 shutdown_connections 2024-03-29T04:08:17.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.469+0000 7fadbed72640 1 --2- 172.21.15.97:0/2279829708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadb806f3f0 0x7fadb8072b80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.469+0000 7fadbed72640 1 --2- 172.21.15.97:0/2279829708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fadb815b370 0x7fadb80b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.469+0000 7fadbed72640 1 -- 172.21.15.97:0/2279829708 >> 172.21.15.97:0/2279829708 conn(0x7fadb806b510 msgr2=0x7fadb806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:17.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.470+0000 7fadbed72640 1 -- 172.21.15.97:0/2279829708 shutdown_connections 2024-03-29T04:08:17.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.470+0000 7fadbed72640 1 -- 172.21.15.97:0/2279829708 wait complete. 2024-03-29T04:08:17.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.471+0000 7fadbed72640 1 Processor -- start 2024-03-29T04:08:17.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.471+0000 7fadbed72640 1 -- start start 2024-03-29T04:08:17.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.471+0000 7fadbed72640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadb806f3f0 0x7fadb814e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.471+0000 7fadbed72640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fadb80730c0 0x7fadb8159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.471+0000 7fadbed72640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadb815b370 0x7fadb81497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.471+0000 7fadbed72640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fadb80b3800 con 0x7fadb815b370 2024-03-29T04:08:17.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.471+0000 7fadbed72640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8073510 con 0x7fadb806f3f0 2024-03-29T04:08:17.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.471+0000 7fadbed72640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fadb80b3980 con 0x7fadb80730c0 2024-03-29T04:08:17.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadbd30f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadb815b370 0x7fadb81497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadbd30f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadb815b370 0x7fadb81497c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34802/0 (socket says 172.21.15.97:34802) 2024-03-29T04:08:17.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadbd30f640 1 -- 172.21.15.97:0/1661961802 learned_addr learned my addr 172.21.15.97:0/1661961802 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:17.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadbcb0e640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadb806f3f0 0x7fadb814e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadb7fff640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fadb80730c0 0x7fadb8159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadbd30f640 1 -- 172.21.15.97:0/1661961802 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fadb80730c0 msgr2=0x7fadb8159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadbd30f640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fadb80730c0 0x7fadb8159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadbd30f640 1 -- 172.21.15.97:0/1661961802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadb806f3f0 msgr2=0x7fadb814e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadbd30f640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadb806f3f0 0x7fadb814e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadbd30f640 1 -- 172.21.15.97:0/1661961802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb814e960 con 0x7fadb815b370 2024-03-29T04:08:17.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadbd30f640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadb815b370 0x7fadb81497c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fadb000c7d0 tx=0x7fadb000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:17.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadb57fa640 1 -- 172.21.15.97:0/1661961802 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0018070 con 0x7fadb815b370 2024-03-29T04:08:17.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadb57fa640 1 -- 172.21.15.97:0/1661961802 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb000a640 con 0x7fadb815b370 2024-03-29T04:08:17.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadb57fa640 1 -- 172.21.15.97:0/1661961802 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb000aa70 con 0x7fadb815b370 2024-03-29T04:08:17.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadbed72640 1 -- 172.21.15.97:0/1661961802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb814ec40 con 0x7fadb815b370 2024-03-29T04:08:17.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.472+0000 7fadbed72640 1 -- 172.21.15.97:0/1661961802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb814f200 con 0x7fadb815b370 2024-03-29T04:08:17.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.473+0000 7fadbed72640 1 -- 172.21.15.97:0/1661961802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb8068640 con 0x7fadb815b370 2024-03-29T04:08:17.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.474+0000 7fadb57fa640 1 -- 172.21.15.97:0/1661961802 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fadb001b070 con 0x7fadb815b370 2024-03-29T04:08:17.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.474+0000 7fadb57fa640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fad9c06c530 0x7fad9c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.474+0000 7fadb57fa640 1 -- 172.21.15.97:0/1661961802 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fadb008ac50 con 0x7fadb815b370 2024-03-29T04:08:17.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.474+0000 7fadbcb0e640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fad9c06c530 0x7fad9c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.474+0000 7fadbcb0e640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fad9c06c530 0x7fad9c06e9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fadac009aa0 tx=0x7fadac009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:17.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.478+0000 7fadb57fa640 1 -- 172.21.15.97:0/1661961802 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fadb0090020 con 0x7fadb815b370 2024-03-29T04:08:17.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.655+0000 7fadbed72640 1 -- 172.21.15.97:0/1661961802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadb80d0000 con 0x7fadb815b370 2024-03-29T04:08:17.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.656+0000 7fadb57fa640 1 -- 172.21.15.97:0/1661961802 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fadb0058c20 con 0x7fadb815b370 2024-03-29T04:08:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:17.636 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.658+0000 7fadbed72640 1 -- 172.21.15.97:0/1661961802 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fad9c06c530 msgr2=0x7fad9c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.658+0000 7fadbed72640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fad9c06c530 0x7fad9c06e9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fadac009aa0 tx=0x7fadac009190 comp rx=0 tx=0).stop 2024-03-29T04:08:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.658+0000 7fadbed72640 1 -- 172.21.15.97:0/1661961802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadb815b370 msgr2=0x7fadb81497c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.658+0000 7fadbed72640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadb815b370 0x7fadb81497c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fadb000c7d0 tx=0x7fadb000cc90 comp rx=0 tx=0).stop 2024-03-29T04:08:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.659+0000 7fadbed72640 1 -- 172.21.15.97:0/1661961802 shutdown_connections 2024-03-29T04:08:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.659+0000 7fadbed72640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fad9c06c530 0x7fad9c06e9f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.659+0000 7fadbed72640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadb815b370 0x7fadb81497c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.659+0000 7fadbed72640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fadb80730c0 0x7fadb8159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.659+0000 7fadbed72640 1 --2- 172.21.15.97:0/1661961802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadb806f3f0 0x7fadb814e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.659+0000 7fadbed72640 1 -- 172.21.15.97:0/1661961802 >> 172.21.15.97:0/1661961802 conn(0x7fadb806b510 msgr2=0x7fadb814f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.659+0000 7fadbed72640 1 -- 172.21.15.97:0/1661961802 shutdown_connections 2024-03-29T04:08:17.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.659+0000 7fadbed72640 1 -- 172.21.15.97:0/1661961802 wait complete. 2024-03-29T04:08:17.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:07:38.618704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:07:39.724339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:08:17.650 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:08:17.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.773+0000 7fe5052a5640 1 Processor -- start 2024-03-29T04:08:17.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.773+0000 7fe5052a5640 1 -- start start 2024-03-29T04:08:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.774+0000 7fe5052a5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5000ce680 0x7fe50009b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.774+0000 7fe5052a5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe50009b140 0x7fe5000cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.774+0000 7fe5052a5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5000d0c80 con 0x7fe5000cea60 2024-03-29T04:08:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.774+0000 7fe5052a5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe5000d0a30 con 0x7fe50009b140 2024-03-29T04:08:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.774+0000 7fe5052a5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe500070650 con 0x7fe5000ce680 2024-03-29T04:08:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.774+0000 7fe4ff7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe50009b140 0x7fe5000cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.774+0000 7fe4ff7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe50009b140 0x7fe5000cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34812/0 (socket says 172.21.15.97:34812) 2024-03-29T04:08:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.774+0000 7fe4ff7fe640 1 -- 172.21.15.97:0/2908039973 learned_addr learned my addr 172.21.15.97:0/2908039973 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.774+0000 7fe4feffd640 1 --2- 172.21.15.97:0/2908039973 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5000ce680 0x7fe50009b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.775+0000 7fe4ff7fe640 1 -- 172.21.15.97:0/2908039973 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5000ce680 msgr2=0x7fe50009b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.775+0000 7fe4ff7fe640 1 --2- 172.21.15.97:0/2908039973 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5000ce680 0x7fe50009b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.775+0000 7fe4ff7fe640 1 -- 172.21.15.97:0/2908039973 >> v1:172.21.15.154:6789/0 conn(0x7fe5000cea60 legacy=0x7fe50009aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.775+0000 7fe4ff7fe640 1 -- 172.21.15.97:0/2908039973 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5000d0f60 con 0x7fe50009b140 2024-03-29T04:08:17.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.775+0000 7fe4feffd640 1 --2- 172.21.15.97:0/2908039973 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5000ce680 0x7fe50009b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:17.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.775+0000 7fe4ff7fe640 1 --2- 172.21.15.97:0/2908039973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50009b140 0x7fe5000cce30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009870 tx=0x7fe4f802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210f82737e3d7519 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:17.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.776+0000 7fe4fdffb640 1 -- 172.21.15.97:0/2908039973 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8035020 con 0x7fe50009b140 2024-03-29T04:08:17.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.776+0000 7fe4fdffb640 1 -- 172.21.15.97:0/2908039973 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f8031640 con 0x7fe50009b140 2024-03-29T04:08:17.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.776+0000 7fe4fdffb640 1 -- 172.21.15.97:0/2908039973 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f802d040 con 0x7fe50009b140 2024-03-29T04:08:17.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.776+0000 7fe5052a5640 1 -- 172.21.15.97:0/2908039973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50009b140 msgr2=0x7fe5000cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.776+0000 7fe5052a5640 1 --2- 172.21.15.97:0/2908039973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50009b140 0x7fe5000cce30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009870 tx=0x7fe4f802b8f0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.776+0000 7fe5052a5640 1 -- 172.21.15.97:0/2908039973 shutdown_connections 2024-03-29T04:08:17.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.776+0000 7fe5052a5640 1 --2- 172.21.15.97:0/2908039973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50009b140 0x7fe5000cce30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.776+0000 7fe5052a5640 1 --2- 172.21.15.97:0/2908039973 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5000ce680 0x7fe50009b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.776+0000 7fe5052a5640 1 -- 172.21.15.97:0/2908039973 >> 172.21.15.97:0/2908039973 conn(0x7fe5000b3f40 msgr2=0x7fe5000b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:17.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.776+0000 7fe5052a5640 1 -- 172.21.15.97:0/2908039973 shutdown_connections 2024-03-29T04:08:17.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.776+0000 7fe5052a5640 1 -- 172.21.15.97:0/2908039973 wait complete. 2024-03-29T04:08:17.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.777+0000 7fe5052a5640 1 Processor -- start 2024-03-29T04:08:17.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.777+0000 7fe5052a5640 1 -- start start 2024-03-29T04:08:17.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe5052a5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe50009b140 0x7fe50006df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe5052a5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5000ce680 0x7fe5001599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe5052a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5000cea60 0x7fe5000b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe5052a5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe500070ae0 con 0x7fe5000cea60 2024-03-29T04:08:17.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe5052a5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe50009bcb0 con 0x7fe5000ce680 2024-03-29T04:08:17.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe5052a5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe500070c60 con 0x7fe50009b140 2024-03-29T04:08:17.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe4ff7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5000cea60 0x7fe5000b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe4ff7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5000cea60 0x7fe5000b6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34814/0 (socket says 172.21.15.97:34814) 2024-03-29T04:08:17.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe4ff7fe640 1 -- 172.21.15.97:0/3098909497 learned_addr learned my addr 172.21.15.97:0/3098909497 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:17.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe4feffd640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe50009b140 0x7fe50006df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe4fe7fc640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5000ce680 0x7fe5001599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe4ff7fe640 1 -- 172.21.15.97:0/3098909497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe50009b140 msgr2=0x7fe50006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe4ff7fe640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe50009b140 0x7fe50006df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe4ff7fe640 1 -- 172.21.15.97:0/3098909497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5000ce680 msgr2=0x7fe5001599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe4ff7fe640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5000ce680 0x7fe5001599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.778+0000 7fe4ff7fe640 1 -- 172.21.15.97:0/3098909497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5000b5280 con 0x7fe5000cea60 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.779+0000 7fe4ff7fe640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5000cea60 0x7fe5000b6320 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009870 tx=0x7fe4f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.779+0000 7fe4e77fe640 1 -- 172.21.15.97:0/3098909497 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f803aa60 con 0x7fe5000cea60 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.779+0000 7fe4e77fe640 1 -- 172.21.15.97:0/3098909497 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f803ac20 con 0x7fe5000cea60 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.779+0000 7fe5052a5640 1 -- 172.21.15.97:0/3098909497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5000b5560 con 0x7fe5000cea60 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.779+0000 7fe4e77fe640 1 -- 172.21.15.97:0/3098909497 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f80443f0 con 0x7fe5000cea60 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.779+0000 7fe5052a5640 1 -- 172.21.15.97:0/3098909497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5000b5b20 con 0x7fe5000cea60 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.780+0000 7fe5052a5640 1 -- 172.21.15.97:0/3098909497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5000b4ff0 con 0x7fe5000cea60 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.780+0000 7fe4e77fe640 1 -- 172.21.15.97:0/3098909497 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe4f8038020 con 0x7fe5000cea60 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.781+0000 7fe4e77fe640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe4ec06c530 0x7fe4ec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.781+0000 7fe4e77fe640 1 -- 172.21.15.97:0/3098909497 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe4f80ad280 con 0x7fe5000cea60 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.781+0000 7fe4feffd640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe4ec06c530 0x7fe4ec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:17.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.781+0000 7fe4feffd640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe4ec06c530 0x7fe4ec06e9f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe5001f8350 tx=0x7fe4f0009890 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:17.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.785+0000 7fe4e77fe640 1 -- 172.21.15.97:0/3098909497 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe4f803f070 con 0x7fe5000cea60 2024-03-29T04:08:17.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.961+0000 7fe5052a5640 1 -- 172.21.15.97:0/3098909497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe5000a0520 con 0x7fe5000cea60 2024-03-29T04:08:17.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.961+0000 7fe4e77fe640 1 -- 172.21.15.97:0/3098909497 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fe4f807b2c0 con 0x7fe5000cea60 2024-03-29T04:08:17.941 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:08:17.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.963+0000 7fe5052a5640 1 -- 172.21.15.97:0/3098909497 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe4ec06c530 msgr2=0x7fe4ec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.963+0000 7fe5052a5640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe4ec06c530 0x7fe4ec06e9f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe5001f8350 tx=0x7fe4f0009890 comp rx=0 tx=0).stop 2024-03-29T04:08:17.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.963+0000 7fe5052a5640 1 -- 172.21.15.97:0/3098909497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5000cea60 msgr2=0x7fe5000b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.963+0000 7fe5052a5640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5000cea60 0x7fe5000b6320 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009870 tx=0x7fe4f80046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.963+0000 7fe5052a5640 1 -- 172.21.15.97:0/3098909497 shutdown_connections 2024-03-29T04:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.963+0000 7fe5052a5640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe4ec06c530 0x7fe4ec06e9f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.963+0000 7fe5052a5640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5000cea60 0x7fe5000b6320 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.963+0000 7fe5052a5640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5000ce680 0x7fe5001599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.963+0000 7fe5052a5640 1 --2- 172.21.15.97:0/3098909497 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe50009b140 0x7fe50006df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.963+0000 7fe5052a5640 1 -- 172.21.15.97:0/3098909497 >> 172.21.15.97:0/3098909497 conn(0x7fe5000b3f40 msgr2=0x7fe5000b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.963+0000 7fe5052a5640 1 -- 172.21.15.97:0/3098909497 shutdown_connections 2024-03-29T04:08:17.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:17.963+0000 7fe5052a5640 1 -- 172.21.15.97:0/3098909497 wait complete. 2024-03-29T04:08:17.954 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:08:18.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.077+0000 7f94fa98d640 1 Processor -- start 2024-03-29T04:08:18.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.077+0000 7f94fa98d640 1 -- start start 2024-03-29T04:08:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.077+0000 7f94fa98d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f94f415e830 0x7f94f406c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.077+0000 7f94fa98d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94f406ca60 0x7f94f406e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.077+0000 7f94fa98d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f94f406e630 0x7f94f4162490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.077+0000 7f94fa98d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f94f406cec0 con 0x7f94f406e630 2024-03-29T04:08:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.077+0000 7f94fa98d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f94f406eac0 con 0x7f94f415e830 2024-03-29T04:08:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.077+0000 7f94fa98d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f94f406ed90 con 0x7f94f406ca60 2024-03-29T04:08:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.078+0000 7f94f8f2a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f94f406e630 0x7f94f4162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.078+0000 7f94f8f2a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f94f406e630 0x7f94f4162490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34816/0 (socket says 172.21.15.97:34816) 2024-03-29T04:08:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.078+0000 7f94f8f2a640 1 -- 172.21.15.97:0/2147407339 learned_addr learned my addr 172.21.15.97:0/2147407339 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.078+0000 7f94f37fe640 1 --2- 172.21.15.97:0/2147407339 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94f406ca60 0x7f94f406e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.078+0000 7f94f3fff640 1 --2- 172.21.15.97:0/2147407339 >> v2:172.21.15.154:3300/0 conn(0x7f94f415e830 0x7f94f406c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.078+0000 7f94f8f2a640 1 -- 172.21.15.97:0/2147407339 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94f406ca60 msgr2=0x7f94f406e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.078+0000 7f94f8f2a640 1 --2- 172.21.15.97:0/2147407339 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94f406ca60 0x7f94f406e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.078+0000 7f94f8f2a640 1 -- 172.21.15.97:0/2147407339 >> v2:172.21.15.154:3300/0 conn(0x7f94f415e830 msgr2=0x7f94f406c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.078+0000 7f94f8f2a640 1 --2- 172.21.15.97:0/2147407339 >> v2:172.21.15.154:3300/0 conn(0x7f94f415e830 0x7f94f406c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.078+0000 7f94f8f2a640 1 -- 172.21.15.97:0/2147407339 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f406f070 con 0x7f94f406e630 2024-03-29T04:08:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.079+0000 7f94f8f2a640 1 --2- 172.21.15.97:0/2147407339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f406e630 0x7f94f4162490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f94f4073d90 tx=0x7f94ec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6901016bde3d83a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.079+0000 7f94f2ffd640 1 -- 172.21.15.97:0/2147407339 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec02ca70 con 0x7f94f406e630 2024-03-29T04:08:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.079+0000 7f94f2ffd640 1 -- 172.21.15.97:0/2147407339 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94ec02cc10 con 0x7f94f406e630 2024-03-29T04:08:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.079+0000 7f94f2ffd640 1 -- 172.21.15.97:0/2147407339 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec0343f0 con 0x7f94f406e630 2024-03-29T04:08:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.079+0000 7f94fa98d640 1 -- 172.21.15.97:0/2147407339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f406e630 msgr2=0x7f94f4162490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.079+0000 7f94fa98d640 1 --2- 172.21.15.97:0/2147407339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f406e630 0x7f94f4162490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f94f4073d90 tx=0x7f94ec02bed0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.079+0000 7f94fa98d640 1 -- 172.21.15.97:0/2147407339 shutdown_connections 2024-03-29T04:08:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.079+0000 7f94fa98d640 1 --2- 172.21.15.97:0/2147407339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f406e630 0x7f94f4162490 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.079+0000 7f94fa98d640 1 --2- 172.21.15.97:0/2147407339 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94f406ca60 0x7f94f406e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.079+0000 7f94fa98d640 1 --2- 172.21.15.97:0/2147407339 >> v2:172.21.15.154:3300/0 conn(0x7f94f415e830 0x7f94f406c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.079+0000 7f94fa98d640 1 -- 172.21.15.97:0/2147407339 >> 172.21.15.97:0/2147407339 conn(0x7f94f4072090 msgr2=0x7f94f40724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.079+0000 7f94fa98d640 1 -- 172.21.15.97:0/2147407339 shutdown_connections 2024-03-29T04:08:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.079+0000 7f94fa98d640 1 -- 172.21.15.97:0/2147407339 wait complete. 2024-03-29T04:08:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.080+0000 7f94fa98d640 1 Processor -- start 2024-03-29T04:08:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.081+0000 7f94fa98d640 1 -- start start 2024-03-29T04:08:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.081+0000 7f94fa98d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f406ca60 0x7f94f414d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.081+0000 7f94fa98d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94f406e630 0x7f94f40c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.081+0000 7f94fa98d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94f415e830 0x7f94f4149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.081+0000 7f94fa98d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f94f406f910 con 0x7f94f406ca60 2024-03-29T04:08:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.081+0000 7f94fa98d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f94f406f630 con 0x7f94f415e830 2024-03-29T04:08:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.081+0000 7f94fa98d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f94f415ec10 con 0x7f94f406e630 2024-03-29T04:08:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.081+0000 7f94f3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f406ca60 0x7f94f414d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.081+0000 7f94f3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f406ca60 0x7f94f414d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34824/0 (socket says 172.21.15.97:34824) 2024-03-29T04:08:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.081+0000 7f94f3fff640 1 -- 172.21.15.97:0/2727250938 learned_addr learned my addr 172.21.15.97:0/2727250938 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.082+0000 7f94f8f2a640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94f415e830 0x7f94f4149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.082+0000 7f94f37fe640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94f406e630 0x7f94f40c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.082+0000 7f94f3fff640 1 -- 172.21.15.97:0/2727250938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94f406e630 msgr2=0x7f94f40c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.082+0000 7f94f3fff640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94f406e630 0x7f94f40c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.082+0000 7f94f3fff640 1 -- 172.21.15.97:0/2727250938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94f415e830 msgr2=0x7f94f4149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.082+0000 7f94f3fff640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94f415e830 0x7f94f4149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.082+0000 7f94f3fff640 1 -- 172.21.15.97:0/2727250938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f40c6030 con 0x7f94f406ca60 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.082+0000 7f94f3fff640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f406ca60 0x7f94f414d2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f94e400e870 tx=0x7f94e400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.082+0000 7f94f0ff9640 1 -- 172.21.15.97:0/2727250938 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4017070 con 0x7f94f406ca60 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.082+0000 7f94f0ff9640 1 -- 172.21.15.97:0/2727250938 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e400c940 con 0x7f94f406ca60 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.082+0000 7f94fa98d640 1 -- 172.21.15.97:0/2727250938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f4149e00 con 0x7f94f406ca60 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.082+0000 7f94f0ff9640 1 -- 172.21.15.97:0/2727250938 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e400cd30 con 0x7f94f406ca60 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.082+0000 7f94fa98d640 1 -- 172.21.15.97:0/2727250938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f414a3c0 con 0x7f94f406ca60 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.083+0000 7f94fa98d640 1 -- 172.21.15.97:0/2727250938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f4162200 con 0x7f94f406ca60 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.084+0000 7f94f0ff9640 1 -- 172.21.15.97:0/2727250938 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f94e401a070 con 0x7f94f406ca60 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.084+0000 7f94f0ff9640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f94e006c530 0x7f94e006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.084+0000 7f94f0ff9640 1 -- 172.21.15.97:0/2727250938 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f94e408b7e0 con 0x7f94f406ca60 2024-03-29T04:08:18.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.084+0000 7f94f37fe640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f94e006c530 0x7f94e006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.084+0000 7f94f37fe640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f94e006c530 0x7f94e006e9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f94f415d9a0 tx=0x7f94e8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.088+0000 7f94f0ff9640 1 -- 172.21.15.97:0/2727250938 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f94e4059840 con 0x7f94f406ca60 2024-03-29T04:08:18.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.262+0000 7f94fa98d640 1 -- 172.21.15.97:0/2727250938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f94f404f9d0 con 0x7f94f406ca60 2024-03-29T04:08:18.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.262+0000 7f94f0ff9640 1 -- 172.21.15.97:0/2727250938 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f94e405db40 con 0x7f94f406ca60 2024-03-29T04:08:18.242 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:08:18.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.264+0000 7f94fa98d640 1 -- 172.21.15.97:0/2727250938 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f94e006c530 msgr2=0x7f94e006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.264+0000 7f94fa98d640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f94e006c530 0x7f94e006e9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f94f415d9a0 tx=0x7f94e8009190 comp rx=0 tx=0).stop 2024-03-29T04:08:18.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.264+0000 7f94fa98d640 1 -- 172.21.15.97:0/2727250938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f406ca60 msgr2=0x7f94f414d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.264+0000 7f94fa98d640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f406ca60 0x7f94f414d2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f94e400e870 tx=0x7f94e400ed30 comp rx=0 tx=0).stop 2024-03-29T04:08:18.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.264+0000 7f94fa98d640 1 -- 172.21.15.97:0/2727250938 shutdown_connections 2024-03-29T04:08:18.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.264+0000 7f94fa98d640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f94e006c530 0x7f94e006e9f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.264+0000 7f94fa98d640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94f415e830 0x7f94f4149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.264+0000 7f94fa98d640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f94f406e630 0x7f94f40c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.264+0000 7f94fa98d640 1 --2- 172.21.15.97:0/2727250938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f406ca60 0x7f94f414d2f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.264+0000 7f94fa98d640 1 -- 172.21.15.97:0/2727250938 >> 172.21.15.97:0/2727250938 conn(0x7f94f4072090 msgr2=0x7f94f406c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:18.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.264+0000 7f94fa98d640 1 -- 172.21.15.97:0/2727250938 shutdown_connections 2024-03-29T04:08:18.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.264+0000 7f94fa98d640 1 -- 172.21.15.97:0/2727250938 wait complete. 2024-03-29T04:08:18.255 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:08:18.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.379+0000 7fe9e191c640 1 Processor -- start 2024-03-29T04:08:18.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.379+0000 7fe9e191c640 1 -- start start 2024-03-29T04:08:18.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.379+0000 7fe9e191c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0d16a0 0x7fe9dc0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.379+0000 7fe9e191c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9dc15ed00 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.379+0000 7fe9e191c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9dc0d21e0 con 0x7fe9dc15e920 2024-03-29T04:08:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.379+0000 7fe9e191c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe9dc0d2480 con 0x7fe9dc0d16a0 2024-03-29T04:08:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.379+0000 7fe9da7fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe9dc0cfde0 0x7fe9dc0d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35500/0 (socket says 172.21.15.97:35500) 2024-03-29T04:08:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.379+0000 7fe9da7fc640 1 -- 172.21.15.97:0/2230442870 learned_addr learned my addr 172.21.15.97:0/2230442870 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.380+0000 7fe9db7fe640 1 --2- 172.21.15.97:0/2230442870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0d16a0 0x7fe9dc0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.380+0000 7fe9d9ffb640 1 -- 172.21.15.97:0/2230442870 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 246680212 0 0) 0x7fe9dc15ed00 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.380+0000 7fe9d9ffb640 1 -- 172.21.15.97:0/2230442870 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9c8003520 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.380+0000 7fe9d9ffb640 1 -- 172.21.15.97:0/2230442870 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4043157510 0 0) 0x7fe9dc0d21e0 con 0x7fe9dc15e920 2024-03-29T04:08:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.380+0000 7fe9d9ffb640 1 -- 172.21.15.97:0/2230442870 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9dc15ed00 con 0x7fe9dc15e920 2024-03-29T04:08:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.380+0000 7fe9db7fe640 1 -- 172.21.15.97:0/2230442870 >> v1:172.21.15.154:6789/0 conn(0x7fe9dc15e920 legacy=0x7fe9dc0d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.380+0000 7fe9db7fe640 1 -- 172.21.15.97:0/2230442870 >> v1:172.21.15.97:6789/0 conn(0x7fe9dc0cfde0 legacy=0x7fe9dc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.380+0000 7fe9db7fe640 1 -- 172.21.15.97:0/2230442870 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc0d2760 con 0x7fe9dc0d16a0 2024-03-29T04:08:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.381+0000 7fe9db7fe640 1 --2- 172.21.15.97:0/2230442870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0d16a0 0x7fe9dc0d0240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40097d0 tx=0x7fe9d402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c714a4e91c6fb33 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:18.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.381+0000 7fe9d9ffb640 1 -- 172.21.15.97:0/2230442870 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4035020 con 0x7fe9dc0d16a0 2024-03-29T04:08:18.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.381+0000 7fe9d9ffb640 1 -- 172.21.15.97:0/2230442870 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d4031890 con 0x7fe9dc0d16a0 2024-03-29T04:08:18.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.381+0000 7fe9d9ffb640 1 -- 172.21.15.97:0/2230442870 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4031c40 con 0x7fe9dc0d16a0 2024-03-29T04:08:18.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.381+0000 7fe9e191c640 1 -- 172.21.15.97:0/2230442870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0d16a0 msgr2=0x7fe9dc0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.381+0000 7fe9e191c640 1 --2- 172.21.15.97:0/2230442870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0d16a0 0x7fe9dc0d0240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40097d0 tx=0x7fe9d402b930 comp rx=0 tx=0).stop 2024-03-29T04:08:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.381+0000 7fe9e191c640 1 -- 172.21.15.97:0/2230442870 shutdown_connections 2024-03-29T04:08:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.381+0000 7fe9e191c640 1 --2- 172.21.15.97:0/2230442870 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc0d16a0 0x7fe9dc0d0240 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.381+0000 7fe9e191c640 1 -- 172.21.15.97:0/2230442870 >> 172.21.15.97:0/2230442870 conn(0x7fe9dc08bfc0 msgr2=0x7fe9dc0d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.381+0000 7fe9e191c640 1 -- 172.21.15.97:0/2230442870 shutdown_connections 2024-03-29T04:08:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.381+0000 7fe9e191c640 1 -- 172.21.15.97:0/2230442870 wait complete. 2024-03-29T04:08:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.382+0000 7fe9e191c640 1 Processor -- start 2024-03-29T04:08:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.383+0000 7fe9e191c640 1 -- start start 2024-03-29T04:08:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.383+0000 7fe9e191c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9dc0cfde0 0x7fe9dc14e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.383+0000 7fe9e191c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9dc0d16a0 0x7fe9dc0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.383+0000 7fe9e191c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc15e920 0x7fe9dc15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.383+0000 7fe9e191c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc0d4ef0 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.383+0000 7fe9e191c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc0d2d20 con 0x7fe9dc0d16a0 2024-03-29T04:08:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.383+0000 7fe9e191c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe9dc0d5070 con 0x7fe9dc15e920 2024-03-29T04:08:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.383+0000 7fe9daffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9dc0cfde0 0x7fe9dc14e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.383+0000 7fe9daffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9dc0cfde0 0x7fe9dc14e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34830/0 (socket says 172.21.15.97:34830) 2024-03-29T04:08:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.383+0000 7fe9daffd640 1 -- 172.21.15.97:0/814158372 learned_addr learned my addr 172.21.15.97:0/814158372 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.384+0000 7fe9da7fc640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9dc0d16a0 0x7fe9dc0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.384+0000 7fe9db7fe640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc15e920 0x7fe9dc15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.384+0000 7fe9daffd640 1 -- 172.21.15.97:0/814158372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc15e920 msgr2=0x7fe9dc15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.384+0000 7fe9daffd640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc15e920 0x7fe9dc15b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.384+0000 7fe9daffd640 1 -- 172.21.15.97:0/814158372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9dc0d16a0 msgr2=0x7fe9dc0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.384+0000 7fe9daffd640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9dc0d16a0 0x7fe9dc0c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.384+0000 7fe9daffd640 1 -- 172.21.15.97:0/814158372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc0c8bc0 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.384+0000 7fe9daffd640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9dc0cfde0 0x7fe9dc14e8b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00c7a0 tx=0x7fe9cc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.384+0000 7fe9d3fff640 1 -- 172.21.15.97:0/814158372 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc003b70 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.384+0000 7fe9d3fff640 1 -- 172.21.15.97:0/814158372 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9cc003d30 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.384+0000 7fe9e191c640 1 -- 172.21.15.97:0/814158372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9dc0c8ea0 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.384+0000 7fe9d3fff640 1 -- 172.21.15.97:0/814158372 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc00aa30 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.384+0000 7fe9e191c640 1 -- 172.21.15.97:0/814158372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9dc0c99a0 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.385+0000 7fe9e191c640 1 -- 172.21.15.97:0/814158372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9dc0a2560 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.385+0000 7fe9d3fff640 1 -- 172.21.15.97:0/814158372 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe9cc00abd0 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.386+0000 7fe9d3fff640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe9bc06c530 0x7fe9bc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.386+0000 7fe9da7fc640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe9bc06c530 0x7fe9bc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.386+0000 7fe9d3fff640 1 -- 172.21.15.97:0/814158372 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe9cc08b3b0 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.386+0000 7fe9da7fc640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe9bc06c530 0x7fe9bc06e9f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0b2d90 tx=0x7fe9c4007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:18.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.390+0000 7fe9d3fff640 1 -- 172.21.15.97:0/814158372 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe9cc059410 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.564+0000 7fe9e191c640 1 -- 172.21.15.97:0/814158372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe9dc04f9d0 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.565+0000 7fe9d3fff640 1 -- 172.21.15.97:0/814158372 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fe9cc05d710 con 0x7fe9dc0cfde0 2024-03-29T04:08:18.545 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:08:18.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.567+0000 7fe9e191c640 1 -- 172.21.15.97:0/814158372 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe9bc06c530 msgr2=0x7fe9bc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.567+0000 7fe9e191c640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe9bc06c530 0x7fe9bc06e9f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0b2d90 tx=0x7fe9c4007d30 comp rx=0 tx=0).stop 2024-03-29T04:08:18.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.567+0000 7fe9e191c640 1 -- 172.21.15.97:0/814158372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9dc0cfde0 msgr2=0x7fe9dc14e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.567+0000 7fe9e191c640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9dc0cfde0 0x7fe9dc14e8b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00c7a0 tx=0x7fe9cc00cc60 comp rx=0 tx=0).stop 2024-03-29T04:08:18.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.567+0000 7fe9e191c640 1 -- 172.21.15.97:0/814158372 shutdown_connections 2024-03-29T04:08:18.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.567+0000 7fe9e191c640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe9bc06c530 0x7fe9bc06e9f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.567+0000 7fe9e191c640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe9dc15e920 0x7fe9dc15b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.567+0000 7fe9e191c640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9dc0d16a0 0x7fe9dc0c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.567+0000 7fe9e191c640 1 --2- 172.21.15.97:0/814158372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9dc0cfde0 0x7fe9dc14e8b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.567+0000 7fe9e191c640 1 -- 172.21.15.97:0/814158372 >> 172.21.15.97:0/814158372 conn(0x7fe9dc08bfc0 msgr2=0x7fe9dc0d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.567+0000 7fe9e191c640 1 -- 172.21.15.97:0/814158372 shutdown_connections 2024-03-29T04:08:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.567+0000 7fe9e191c640 1 -- 172.21.15.97:0/814158372 wait complete. 2024-03-29T04:08:18.557 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:08:18.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.678+0000 7f505c5bd640 1 Processor -- start 2024-03-29T04:08:18.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.678+0000 7f505c5bd640 1 -- start start 2024-03-29T04:08:18.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.679+0000 7f505c5bd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f505406f1e0 0x7f50540b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.679+0000 7f505c5bd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50540b3c10 con 0x7f505406e1c0 2024-03-29T04:08:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.679+0000 7f505c5bd640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50540b3ef0 con 0x7f5054144790 2024-03-29T04:08:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.679+0000 7f505c5bd640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f50540b41d0 con 0x7f505406f1e0 2024-03-29T04:08:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.679+0000 7f5059b58640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f505406f1e0 0x7f50540b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.679+0000 7f505ab5a640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f505406e1c0 0x7f50540ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35504/0 (socket says 172.21.15.97:35504) 2024-03-29T04:08:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.679+0000 7f505ab5a640 1 -- 172.21.15.97:0/3459024343 learned_addr learned my addr 172.21.15.97:0/3459024343 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.679+0000 7f5059b58640 1 -- 172.21.15.97:0/3459024343 >> v1:172.21.15.154:6789/0 conn(0x7f5054144790 legacy=0x7f50540b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.679+0000 7f5059b58640 1 -- 172.21.15.97:0/3459024343 >> v1:172.21.15.97:6789/0 conn(0x7f505406e1c0 legacy=0x7f50540ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.679+0000 7f5059357640 1 -- 172.21.15.97:0/3459024343 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1760984143 0 0) 0x7f50540b3ef0 con 0x7f5054144790 2024-03-29T04:08:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.679+0000 7f5059b58640 1 -- 172.21.15.97:0/3459024343 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50540b44b0 con 0x7f505406f1e0 2024-03-29T04:08:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.680+0000 7f5059b58640 1 --2- 172.21.15.97:0/3459024343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f505406f1e0 0x7f50540b32c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f504400ba40 tx=0x7f504402dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49e80dcb1c4cb49 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:18.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.680+0000 7f5059357640 1 -- 172.21.15.97:0/3459024343 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044034070 con 0x7f505406f1e0 2024-03-29T04:08:18.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.680+0000 7f5059357640 1 -- 172.21.15.97:0/3459024343 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f504402ea70 con 0x7f505406f1e0 2024-03-29T04:08:18.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.680+0000 7f5059357640 1 -- 172.21.15.97:0/3459024343 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504402ee40 con 0x7f505406f1e0 2024-03-29T04:08:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.680+0000 7f505c5bd640 1 -- 172.21.15.97:0/3459024343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f505406f1e0 msgr2=0x7f50540b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.680+0000 7f505c5bd640 1 --2- 172.21.15.97:0/3459024343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f505406f1e0 0x7f50540b32c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f504400ba40 tx=0x7f504402dd40 comp rx=0 tx=0).stop 2024-03-29T04:08:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.681+0000 7f505c5bd640 1 -- 172.21.15.97:0/3459024343 shutdown_connections 2024-03-29T04:08:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.681+0000 7f505c5bd640 1 --2- 172.21.15.97:0/3459024343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f505406f1e0 0x7f50540b32c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.681+0000 7f505c5bd640 1 -- 172.21.15.97:0/3459024343 >> 172.21.15.97:0/3459024343 conn(0x7f505406b510 msgr2=0x7f505406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.681+0000 7f505c5bd640 1 -- 172.21.15.97:0/3459024343 shutdown_connections 2024-03-29T04:08:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.681+0000 7f505c5bd640 1 -- 172.21.15.97:0/3459024343 wait complete. 2024-03-29T04:08:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.682+0000 7f505c5bd640 1 Processor -- start 2024-03-29T04:08:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.682+0000 7f505c5bd640 1 -- start start 2024-03-29T04:08:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.682+0000 7f505c5bd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f505406e1c0 0x7f5054157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.682+0000 7f505c5bd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f505406f1e0 0x7f505409a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.682+0000 7f505c5bd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5054144790 0x7f505409a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.682+0000 7f505c5bd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f50540b3980 con 0x7f5054144790 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.682+0000 7f505c5bd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f50540b3800 con 0x7f505406f1e0 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.682+0000 7f505c5bd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f50540ade00 con 0x7f505406e1c0 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f505ab5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5054144790 0x7f505409a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f505ab5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5054144790 0x7f505409a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34846/0 (socket says 172.21.15.97:34846) 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f505ab5a640 1 -- 172.21.15.97:0/2422652653 learned_addr learned my addr 172.21.15.97:0/2422652653 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f505a359640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f505406e1c0 0x7f5054157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f5059b58640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f505406f1e0 0x7f505409a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f505ab5a640 1 -- 172.21.15.97:0/2422652653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f505406e1c0 msgr2=0x7f5054157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f505ab5a640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f505406e1c0 0x7f5054157450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f505ab5a640 1 -- 172.21.15.97:0/2422652653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f505406f1e0 msgr2=0x7f505409a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f505ab5a640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f505406f1e0 0x7f505409a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f505ab5a640 1 -- 172.21.15.97:0/2422652653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50540b15f0 con 0x7f5054144790 2024-03-29T04:08:18.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f505ab5a640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5054144790 0x7f505409a9b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f504c00caf0 tx=0x7f504c0020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f504affd640 1 -- 172.21.15.97:0/2422652653 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c018070 con 0x7f5054144790 2024-03-29T04:08:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f504affd640 1 -- 172.21.15.97:0/2422652653 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f504c00a700 con 0x7f5054144790 2024-03-29T04:08:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f505c5bd640 1 -- 172.21.15.97:0/2422652653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505409afe0 con 0x7f5054144790 2024-03-29T04:08:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f504affd640 1 -- 172.21.15.97:0/2422652653 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c00aaf0 con 0x7f5054144790 2024-03-29T04:08:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.683+0000 7f505c5bd640 1 -- 172.21.15.97:0/2422652653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f505409d3f0 con 0x7f5054144790 2024-03-29T04:08:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.684+0000 7f505c5bd640 1 -- 172.21.15.97:0/2422652653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5054068640 con 0x7f5054144790 2024-03-29T04:08:18.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.685+0000 7f504affd640 1 -- 172.21.15.97:0/2422652653 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f504c01b070 con 0x7f5054144790 2024-03-29T04:08:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.685+0000 7f504affd640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f503806c530 0x7f503806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.685+0000 7f504affd640 1 -- 172.21.15.97:0/2422652653 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f504c08ad80 con 0x7f5054144790 2024-03-29T04:08:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.685+0000 7f505a359640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f503806c530 0x7f503806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.685+0000 7f505a359640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f503806c530 0x7f503806e9f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5050000f60 tx=0x7f5050009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:18.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.689+0000 7f504affd640 1 -- 172.21.15.97:0/2422652653 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f504c058d50 con 0x7f5054144790 2024-03-29T04:08:18.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.903+0000 7f505c5bd640 1 -- 172.21.15.97:0/2422652653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f505415a6e0 con 0x7f5054144790 2024-03-29T04:08:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.904+0000 7f504affd640 1 -- 172.21.15.97:0/2422652653 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7f504c05d050 con 0x7f5054144790 2024-03-29T04:08:18.884 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:18.884 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1127043830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127043830},{"type":"v1","addr":"172.21.15.154:6834","nonce":1127043830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3767188750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3767188750},{"type":"v1","addr":"172.21.15.97:6835","nonce":3767188750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3651340478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3651340478},{"type":"v1","addr":"172.21.15.97:6837","nonce":3651340478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:07:40.761466+0000","modified":"2024-03-29T04:07:42.771721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3669418265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3669418265},{"type":"v1","addr":"172.21.15.154:6835","nonce":3669418265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-29T04:08:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-03-29T04:08:18.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.906+0000 7f505c5bd640 1 -- 172.21.15.97:0/2422652653 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f503806c530 msgr2=0x7f503806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.906+0000 7f505c5bd640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f503806c530 0x7f503806e9f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5050000f60 tx=0x7f5050009040 comp rx=0 tx=0).stop 2024-03-29T04:08:18.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.906+0000 7f505c5bd640 1 -- 172.21.15.97:0/2422652653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5054144790 msgr2=0x7f505409a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.906+0000 7f505c5bd640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5054144790 0x7f505409a9b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f504c00caf0 tx=0x7f504c0020f0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.906+0000 7f505c5bd640 1 -- 172.21.15.97:0/2422652653 shutdown_connections 2024-03-29T04:08:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.906+0000 7f505c5bd640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f503806c530 0x7f503806e9f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.906+0000 7f505c5bd640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5054144790 0x7f505409a9b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.906+0000 7f505c5bd640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f505406f1e0 0x7f505409a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.906+0000 7f505c5bd640 1 --2- 172.21.15.97:0/2422652653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f505406e1c0 0x7f5054157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.906+0000 7f505c5bd640 1 -- 172.21.15.97:0/2422652653 >> 172.21.15.97:0/2422652653 conn(0x7f505406b510 msgr2=0x7f50540747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.906+0000 7f505c5bd640 1 -- 172.21.15.97:0/2422652653 shutdown_connections 2024-03-29T04:08:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:18.906+0000 7f505c5bd640 1 -- 172.21.15.97:0/2422652653 wait complete. 2024-03-29T04:08:18.896 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:18.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.016+0000 7f56bda33640 1 Processor -- start 2024-03-29T04:08:18.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.016+0000 7f56bda33640 1 -- start start 2024-03-29T04:08:18.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.017+0000 7f56bda33640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f56b806c520 0x7f56b806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:18.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.017+0000 7f56bda33640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b806e6b0 con 0x7f56b806e0f0 2024-03-29T04:08:18.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.017+0000 7f56bda33640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b806e990 con 0x7f56b815e9a0 2024-03-29T04:08:18.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.017+0000 7f56bda33640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f56b806f2c0 con 0x7f56b806c520 2024-03-29T04:08:18.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.017+0000 7f56b67fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f56b806e0f0 0x7f56b806f750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35518/0 (socket says 172.21.15.97:35518) 2024-03-29T04:08:18.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.018+0000 7f56b67fc640 1 -- 172.21.15.97:0/4065662767 learned_addr learned my addr 172.21.15.97:0/4065662767 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:18.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.018+0000 7f56b77fe640 1 --2- 172.21.15.97:0/4065662767 >> v2:172.21.15.154:3300/0 conn(0x7f56b806c520 0x7f56b806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:18.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.018+0000 7f56b5ffb640 1 -- 172.21.15.97:0/4065662767 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 347669805 0 0) 0x7f56b806e6b0 con 0x7f56b806e0f0 2024-03-29T04:08:18.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.018+0000 7f56b5ffb640 1 -- 172.21.15.97:0/4065662767 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56a0003520 con 0x7f56b806e0f0 2024-03-29T04:08:18.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.018+0000 7f56b77fe640 1 -- 172.21.15.97:0/4065662767 >> v1:172.21.15.154:6789/0 conn(0x7f56b815e9a0 legacy=0x7f56b8072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.018+0000 7f56b5ffb640 1 -- 172.21.15.97:0/4065662767 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3918553284 0 0) 0x7f56b806e990 con 0x7f56b815e9a0 2024-03-29T04:08:18.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.018+0000 7f56b77fe640 1 -- 172.21.15.97:0/4065662767 >> v1:172.21.15.97:6789/0 conn(0x7f56b806e0f0 legacy=0x7f56b806f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:18.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.018+0000 7f56b77fe640 1 -- 172.21.15.97:0/4065662767 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b806ec70 con 0x7f56b806c520 2024-03-29T04:08:19.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.019+0000 7f56b77fe640 1 --2- 172.21.15.97:0/4065662767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b806c520 0x7f56b806c9b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f56b00097a0 tx=0x7f56b002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c689efdef9a1cf9a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:19.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.019+0000 7f56b5ffb640 1 -- 172.21.15.97:0/4065662767 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0035020 con 0x7f56b806c520 2024-03-29T04:08:19.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.019+0000 7f56b5ffb640 1 -- 172.21.15.97:0/4065662767 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b0031830 con 0x7f56b806c520 2024-03-29T04:08:19.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.019+0000 7f56b5ffb640 1 -- 172.21.15.97:0/4065662767 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0031be0 con 0x7f56b806c520 2024-03-29T04:08:19.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.019+0000 7f56bda33640 1 -- 172.21.15.97:0/4065662767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b806c520 msgr2=0x7f56b806c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.019+0000 7f56bda33640 1 --2- 172.21.15.97:0/4065662767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b806c520 0x7f56b806c9b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f56b00097a0 tx=0x7f56b002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.019+0000 7f56bda33640 1 -- 172.21.15.97:0/4065662767 shutdown_connections 2024-03-29T04:08:19.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.019+0000 7f56bda33640 1 --2- 172.21.15.97:0/4065662767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b806c520 0x7f56b806c9b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.019+0000 7f56bda33640 1 -- 172.21.15.97:0/4065662767 >> 172.21.15.97:0/4065662767 conn(0x7f56b8072090 msgr2=0x7f56b80724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:19.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.019+0000 7f56bda33640 1 -- 172.21.15.97:0/4065662767 shutdown_connections 2024-03-29T04:08:19.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.019+0000 7f56bda33640 1 -- 172.21.15.97:0/4065662767 wait complete. 2024-03-29T04:08:19.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.020+0000 7f56bda33640 1 Processor -- start 2024-03-29T04:08:19.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.021+0000 7f56bda33640 1 -- start start 2024-03-29T04:08:19.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.021+0000 7f56bda33640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56b806c520 0x7f56b81585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.021+0000 7f56bda33640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b806e0f0 0x7f56b815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.021+0000 7f56bda33640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b815e9a0 0x7f56b80b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.021+0000 7f56bda33640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56b80ada80 con 0x7f56b806c520 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.021+0000 7f56bda33640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f56b806fe60 con 0x7f56b815e9a0 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.021+0000 7f56bda33640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f56b80adc00 con 0x7f56b806e0f0 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.021+0000 7f56b6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56b806c520 0x7f56b81585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.021+0000 7f56b6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56b806c520 0x7f56b81585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34848/0 (socket says 172.21.15.97:34848) 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.021+0000 7f56b6ffd640 1 -- 172.21.15.97:0/1944976274 learned_addr learned my addr 172.21.15.97:0/1944976274 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.022+0000 7f56b77fe640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b815e9a0 0x7f56b80b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.022+0000 7f56b67fc640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b806e0f0 0x7f56b815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.022+0000 7f56b6ffd640 1 -- 172.21.15.97:0/1944976274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b806e0f0 msgr2=0x7f56b815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.022+0000 7f56b6ffd640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b806e0f0 0x7f56b815c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.022+0000 7f56b6ffd640 1 -- 172.21.15.97:0/1944976274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b815e9a0 msgr2=0x7f56b80b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.022+0000 7f56b6ffd640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b815e9a0 0x7f56b80b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.022+0000 7f56b6ffd640 1 -- 172.21.15.97:0/1944976274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b815c580 con 0x7f56b806c520 2024-03-29T04:08:19.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.022+0000 7f56b6ffd640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56b806c520 0x7f56b81585a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f56a800c820 tx=0x7f56a800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:19.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.022+0000 7f569ffff640 1 -- 172.21.15.97:0/1944976274 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8017070 con 0x7f56b806c520 2024-03-29T04:08:19.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.022+0000 7f569ffff640 1 -- 172.21.15.97:0/1944976274 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a8003da0 con 0x7f56b806c520 2024-03-29T04:08:19.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.022+0000 7f56bda33640 1 -- 172.21.15.97:0/1944976274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b815c820 con 0x7f56b806c520 2024-03-29T04:08:19.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.022+0000 7f569ffff640 1 -- 172.21.15.97:0/1944976274 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8005290 con 0x7f56b806c520 2024-03-29T04:08:19.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.022+0000 7f56bda33640 1 -- 172.21.15.97:0/1944976274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b815cde0 con 0x7f56b806c520 2024-03-29T04:08:19.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.023+0000 7f56bda33640 1 -- 172.21.15.97:0/1944976274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b8068640 con 0x7f56b806c520 2024-03-29T04:08:19.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.024+0000 7f569ffff640 1 -- 172.21.15.97:0/1944976274 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f56a801a070 con 0x7f56b806c520 2024-03-29T04:08:19.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.024+0000 7f569ffff640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f56a406c530 0x7f56a406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.024+0000 7f569ffff640 1 -- 172.21.15.97:0/1944976274 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f56a808acf0 con 0x7f56b806c520 2024-03-29T04:08:19.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.024+0000 7f56b67fc640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f56a406c530 0x7f56a406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.024+0000 7f56b67fc640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f56a406c530 0x7f56a406e9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f56b80b00d0 tx=0x7f56ac007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:19.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.028+0000 7f569ffff640 1 -- 172.21.15.97:0/1944976274 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f56a808f070 con 0x7f56b806c520 2024-03-29T04:08:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.202+0000 7f56bda33640 1 -- 172.21.15.97:0/1944976274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56b80675d0 con 0x7f56b806c520 2024-03-29T04:08:19.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.203+0000 7f569ffff640 1 -- 172.21.15.97:0/1944976274 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f56a8058cc0 con 0x7f56b806c520 2024-03-29T04:08:19.183 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:19.183 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:19.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.205+0000 7f56bda33640 1 -- 172.21.15.97:0/1944976274 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f56a406c530 msgr2=0x7f56a406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.205+0000 7f56bda33640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f56a406c530 0x7f56a406e9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f56b80b00d0 tx=0x7f56ac007d30 comp rx=0 tx=0).stop 2024-03-29T04:08:19.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.205+0000 7f56bda33640 1 -- 172.21.15.97:0/1944976274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56b806c520 msgr2=0x7f56b81585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.205+0000 7f56bda33640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56b806c520 0x7f56b81585a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f56a800c820 tx=0x7f56a800cce0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.205+0000 7f56bda33640 1 -- 172.21.15.97:0/1944976274 shutdown_connections 2024-03-29T04:08:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.205+0000 7f56bda33640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f56a406c530 0x7f56a406e9f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.205+0000 7f56bda33640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56b815e9a0 0x7f56b80b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.205+0000 7f56bda33640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56b806e0f0 0x7f56b815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.205+0000 7f56bda33640 1 --2- 172.21.15.97:0/1944976274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56b806c520 0x7f56b81585a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.205+0000 7f56bda33640 1 -- 172.21.15.97:0/1944976274 >> 172.21.15.97:0/1944976274 conn(0x7f56b8072090 msgr2=0x7f56b806b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.205+0000 7f56bda33640 1 -- 172.21.15.97:0/1944976274 shutdown_connections 2024-03-29T04:08:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.205+0000 7f56bda33640 1 -- 172.21.15.97:0/1944976274 wait complete. 2024-03-29T04:08:19.196 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T04:08:19.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:08:19.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.316+0000 7fa907da8640 1 Processor -- start 2024-03-29T04:08:19.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.316+0000 7fa907da8640 1 -- start start 2024-03-29T04:08:19.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.317+0000 7fa907da8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa900073290 0x7fa900166430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.317+0000 7fa907da8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa900071ae0 con 0x7fa90006f3f0 2024-03-29T04:08:19.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.317+0000 7fa907da8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa900071dc0 con 0x7fa90015b370 2024-03-29T04:08:19.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.317+0000 7fa907da8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa9000b39d0 con 0x7fa900073290 2024-03-29T04:08:19.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.317+0000 7fa906345640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa900073290 0x7fa900166430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.317+0000 7fa905343640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa90006f3f0 0x7fa900072b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35530/0 (socket says 172.21.15.97:35530) 2024-03-29T04:08:19.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.317+0000 7fa905343640 1 -- 172.21.15.97:0/1752756101 learned_addr learned my addr 172.21.15.97:0/1752756101 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:19.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.317+0000 7fa906345640 1 -- 172.21.15.97:0/1752756101 >> v1:172.21.15.154:6789/0 conn(0x7fa90015b370 legacy=0x7fa9000b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.317+0000 7fa906345640 1 -- 172.21.15.97:0/1752756101 >> v1:172.21.15.97:6789/0 conn(0x7fa90006f3f0 legacy=0x7fa900072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.317+0000 7fa904b42640 1 -- 172.21.15.97:0/1752756101 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 318087519 0 0) 0x7fa900071dc0 con 0x7fa90015b370 2024-03-29T04:08:19.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.317+0000 7fa906345640 1 -- 172.21.15.97:0/1752756101 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9000720a0 con 0x7fa900073290 2024-03-29T04:08:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.318+0000 7fa906345640 1 --2- 172.21.15.97:0/1752756101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa900073290 0x7fa900166430 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009d90 tx=0x7fa8f802bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b9eec596efa67 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.318+0000 7fa904b42640 1 -- 172.21.15.97:0/1752756101 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8039070 con 0x7fa900073290 2024-03-29T04:08:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.318+0000 7fa904b42640 1 -- 172.21.15.97:0/1752756101 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f802cab0 con 0x7fa900073290 2024-03-29T04:08:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.318+0000 7fa904b42640 1 -- 172.21.15.97:0/1752756101 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8034440 con 0x7fa900073290 2024-03-29T04:08:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.319+0000 7fa907da8640 1 -- 172.21.15.97:0/1752756101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa900073290 msgr2=0x7fa900166430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.319+0000 7fa907da8640 1 --2- 172.21.15.97:0/1752756101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa900073290 0x7fa900166430 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009d90 tx=0x7fa8f802bd80 comp rx=0 tx=0).stop 2024-03-29T04:08:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.319+0000 7fa907da8640 1 -- 172.21.15.97:0/1752756101 shutdown_connections 2024-03-29T04:08:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.319+0000 7fa907da8640 1 --2- 172.21.15.97:0/1752756101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa900073290 0x7fa900166430 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.319+0000 7fa907da8640 1 -- 172.21.15.97:0/1752756101 >> 172.21.15.97:0/1752756101 conn(0x7fa90006b510 msgr2=0x7fa90006b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.319+0000 7fa907da8640 1 -- 172.21.15.97:0/1752756101 shutdown_connections 2024-03-29T04:08:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.319+0000 7fa907da8640 1 -- 172.21.15.97:0/1752756101 wait complete. 2024-03-29T04:08:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.320+0000 7fa907da8640 1 Processor -- start 2024-03-29T04:08:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.320+0000 7fa907da8640 1 -- start start 2024-03-29T04:08:19.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa907da8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa90006f3f0 0x7fa900143210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa907da8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa900073290 0x7fa900150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa907da8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa90015b370 0x7fa9000d0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa907da8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa90015b750 con 0x7fa90006f3f0 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa907da8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa900072660 con 0x7fa900073290 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa907da8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa90015b8d0 con 0x7fa90015b370 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa905b44640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa90006f3f0 0x7fa900143210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa905b44640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa90006f3f0 0x7fa900143210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34862/0 (socket says 172.21.15.97:34862) 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa905b44640 1 -- 172.21.15.97:0/1963239342 learned_addr learned my addr 172.21.15.97:0/1963239342 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa906345640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa90015b370 0x7fa9000d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa905343640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa900073290 0x7fa900150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa905b44640 1 -- 172.21.15.97:0/1963239342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa90015b370 msgr2=0x7fa9000d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa905b44640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa90015b370 0x7fa9000d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa905b44640 1 -- 172.21.15.97:0/1963239342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa900073290 msgr2=0x7fa900150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa905b44640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa900073290 0x7fa900150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa905b44640 1 -- 172.21.15.97:0/1963239342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9000cd390 con 0x7fa90006f3f0 2024-03-29T04:08:19.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa905343640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa900073290 0x7fa900150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:19.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa905b44640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa90006f3f0 0x7fa900143210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c4e0 tx=0x7fa8fc00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:19.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa8f67fc640 1 -- 172.21.15.97:0/1963239342 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc017070 con 0x7fa90006f3f0 2024-03-29T04:08:19.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa8f67fc640 1 -- 172.21.15.97:0/1963239342 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8fc0035f0 con 0x7fa90006f3f0 2024-03-29T04:08:19.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa907da8640 1 -- 172.21.15.97:0/1963239342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9000ccdd0 con 0x7fa90006f3f0 2024-03-29T04:08:19.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa8f67fc640 1 -- 172.21.15.97:0/1963239342 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0039e0 con 0x7fa90006f3f0 2024-03-29T04:08:19.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.321+0000 7fa907da8640 1 -- 172.21.15.97:0/1963239342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9000cf200 con 0x7fa90006f3f0 2024-03-29T04:08:19.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.322+0000 7fa907da8640 1 -- 172.21.15.97:0/1963239342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa900068640 con 0x7fa90006f3f0 2024-03-29T04:08:19.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.323+0000 7fa8f67fc640 1 -- 172.21.15.97:0/1963239342 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa8fc01a070 con 0x7fa90006f3f0 2024-03-29T04:08:19.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.323+0000 7fa8f67fc640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa8e406c560 0x7fa8e406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.323+0000 7fa8f67fc640 1 -- 172.21.15.97:0/1963239342 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa8fc08a850 con 0x7fa90006f3f0 2024-03-29T04:08:19.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.323+0000 7fa905343640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa8e406c560 0x7fa8e406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.324+0000 7fa905343640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa8e406c560 0x7fa8e406ea20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa900072ca0 tx=0x7fa8f0007d20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:19.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.328+0000 7fa8f67fc640 1 -- 172.21.15.97:0/1963239342 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa8fc0588b0 con 0x7fa90006f3f0 2024-03-29T04:08:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.538+0000 7fa907da8640 1 -- 172.21.15.97:0/1963239342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9000d0000 con 0x7fa90006f3f0 2024-03-29T04:08:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.539+0000 7fa8f67fc640 1 -- 172.21.15.97:0/1963239342 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7fa8fc05cbb0 con 0x7fa90006f3f0 2024-03-29T04:08:19.519 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:19.519 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1127043830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127043830},{"type":"v1","addr":"172.21.15.154:6834","nonce":1127043830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3767188750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3767188750},{"type":"v1","addr":"172.21.15.97:6835","nonce":3767188750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3651340478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3651340478},{"type":"v1","addr":"172.21.15.97:6837","nonce":3651340478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:07:40.761466+0000","modified":"2024-03-29T04:07:42.771721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3669418265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3669418265},{"type":"v1","addr":"172.21.15.154:6835","nonce":3669418265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-29T04:08:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-03-29T04:08:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.541+0000 7fa907da8640 1 -- 172.21.15.97:0/1963239342 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa8e406c560 msgr2=0x7fa8e406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.541+0000 7fa907da8640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa8e406c560 0x7fa8e406ea20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa900072ca0 tx=0x7fa8f0007d20 comp rx=0 tx=0).stop 2024-03-29T04:08:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.541+0000 7fa907da8640 1 -- 172.21.15.97:0/1963239342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa90006f3f0 msgr2=0x7fa900143210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.541+0000 7fa907da8640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa90006f3f0 0x7fa900143210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c4e0 tx=0x7fa8fc00c9a0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.541+0000 7fa907da8640 1 -- 172.21.15.97:0/1963239342 shutdown_connections 2024-03-29T04:08:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.541+0000 7fa907da8640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa8e406c560 0x7fa8e406ea20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.541+0000 7fa907da8640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa90015b370 0x7fa9000d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.541+0000 7fa907da8640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa900073290 0x7fa900150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.541+0000 7fa907da8640 1 --2- 172.21.15.97:0/1963239342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa90006f3f0 0x7fa900143210 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.541+0000 7fa907da8640 1 -- 172.21.15.97:0/1963239342 >> 172.21.15.97:0/1963239342 conn(0x7fa90006b510 msgr2=0x7fa90006ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.541+0000 7fa907da8640 1 -- 172.21.15.97:0/1963239342 shutdown_connections 2024-03-29T04:08:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.541+0000 7fa907da8640 1 -- 172.21.15.97:0/1963239342 wait complete. 2024-03-29T04:08:19.531 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:08:19.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.651+0000 7f2aed190640 1 Processor -- start 2024-03-29T04:08:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.651+0000 7f2aed190640 1 -- start start 2024-03-29T04:08:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2aed190640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ae815e6e0 0x7f2ae809b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2aed190640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2ae8162ab0 0x7f2ae80b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2aed190640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae80d1a10 con 0x7f2ae80d0660 2024-03-29T04:08:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2aed190640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae80d0e90 con 0x7f2ae8162ab0 2024-03-29T04:08:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2aed190640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae80d1170 con 0x7f2ae815e6e0 2024-03-29T04:08:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2ae6d9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ae815e6e0 0x7f2ae809b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2ae759e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2ae8162ab0 0x7f2ae80b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2adffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2ae80d0660 0x7f2ae81623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:44540/0 (socket says 172.21.15.97:44540) 2024-03-29T04:08:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2adffff640 1 -- 172.21.15.97:0/1978178009 learned_addr learned my addr 172.21.15.97:0/1978178009 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2ae659c640 1 -- 172.21.15.97:0/1978178009 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3215766218 0 0) 0x7f2ae80d1a10 con 0x7f2ae80d0660 2024-03-29T04:08:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2ae6d9d640 1 -- 172.21.15.97:0/1978178009 >> v2:172.21.15.154:3300/0 conn(0x7f2ae8162ab0 msgr2=0x7f2ae80b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2ae6d9d640 1 --2- 172.21.15.97:0/1978178009 >> v2:172.21.15.154:3300/0 conn(0x7f2ae8162ab0 0x7f2ae80b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2ae6d9d640 1 -- 172.21.15.97:0/1978178009 >> v1:172.21.15.154:6789/0 conn(0x7f2ae80d0660 legacy=0x7f2ae81623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.652+0000 7f2ae6d9d640 1 -- 172.21.15.97:0/1978178009 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae80d1450 con 0x7f2ae815e6e0 2024-03-29T04:08:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.653+0000 7f2ae6d9d640 1 --2- 172.21.15.97:0/1978178009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ae815e6e0 0x7f2ae809b6e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80097a0 tx=0x7f2ad802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdd8deb95aac1e0c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.653+0000 7f2ae659c640 1 -- 172.21.15.97:0/1978178009 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8035020 con 0x7f2ae815e6e0 2024-03-29T04:08:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.653+0000 7f2ae659c640 1 -- 172.21.15.97:0/1978178009 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad8031830 con 0x7f2ae815e6e0 2024-03-29T04:08:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.653+0000 7f2ae659c640 1 -- 172.21.15.97:0/1978178009 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8031c00 con 0x7f2ae815e6e0 2024-03-29T04:08:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.654+0000 7f2aed190640 1 -- 172.21.15.97:0/1978178009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ae815e6e0 msgr2=0x7f2ae809b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.654+0000 7f2aed190640 1 --2- 172.21.15.97:0/1978178009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ae815e6e0 0x7f2ae809b6e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80097a0 tx=0x7f2ad802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.654+0000 7f2aed190640 1 -- 172.21.15.97:0/1978178009 shutdown_connections 2024-03-29T04:08:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.654+0000 7f2aed190640 1 --2- 172.21.15.97:0/1978178009 >> v2:172.21.15.154:3300/0 conn(0x7f2ae8162ab0 0x7f2ae80b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.654+0000 7f2aed190640 1 --2- 172.21.15.97:0/1978178009 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ae815e6e0 0x7f2ae809b6e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.654+0000 7f2aed190640 1 -- 172.21.15.97:0/1978178009 >> 172.21.15.97:0/1978178009 conn(0x7f2ae80b3f10 msgr2=0x7f2ae80b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.654+0000 7f2aed190640 1 -- 172.21.15.97:0/1978178009 shutdown_connections 2024-03-29T04:08:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.654+0000 7f2aed190640 1 -- 172.21.15.97:0/1978178009 wait complete. 2024-03-29T04:08:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.655+0000 7f2aed190640 1 Processor -- start 2024-03-29T04:08:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.655+0000 7f2aed190640 1 -- start start 2024-03-29T04:08:19.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2aed190640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ae80d0660 0x7f2ae815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2aed190640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ae815e6e0 0x7f2ae81603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2aed190640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ae8162ab0 0x7f2ae8160920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2aed190640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae806d910 con 0x7f2ae80d0660 2024-03-29T04:08:19.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2aed190640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae80b1d80 con 0x7f2ae8162ab0 2024-03-29T04:08:19.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2aed190640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae80b05b0 con 0x7f2ae815e6e0 2024-03-29T04:08:19.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2ae6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ae80d0660 0x7f2ae815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2ae6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ae80d0660 0x7f2ae815b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34870/0 (socket says 172.21.15.97:34870) 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2ae6d9d640 1 -- 172.21.15.97:0/1107016028 learned_addr learned my addr 172.21.15.97:0/1107016028 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2ae759e640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ae8162ab0 0x7f2ae8160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2adffff640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ae815e6e0 0x7f2ae81603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2ae6d9d640 1 -- 172.21.15.97:0/1107016028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ae815e6e0 msgr2=0x7f2ae81603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2ae6d9d640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ae815e6e0 0x7f2ae81603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2ae6d9d640 1 -- 172.21.15.97:0/1107016028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ae8162ab0 msgr2=0x7f2ae8160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2ae6d9d640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ae8162ab0 0x7f2ae8160920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2ae6d9d640 1 -- 172.21.15.97:0/1107016028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae80b0890 con 0x7f2ae80d0660 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2ae6d9d640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ae80d0660 0x7f2ae815b4d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8000c00 tx=0x7f2ad80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2adf7fe640 1 -- 172.21.15.97:0/1107016028 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8035070 con 0x7f2ae80d0660 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2adf7fe640 1 -- 172.21.15.97:0/1107016028 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad8004230 con 0x7f2ae80d0660 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2aed190640 1 -- 172.21.15.97:0/1107016028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae80b0b70 con 0x7f2ae80d0660 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2adf7fe640 1 -- 172.21.15.97:0/1107016028 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad803a630 con 0x7f2ae80d0660 2024-03-29T04:08:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.656+0000 7f2aed190640 1 -- 172.21.15.97:0/1107016028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae815ba10 con 0x7f2ae80d0660 2024-03-29T04:08:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.657+0000 7f2aed190640 1 -- 172.21.15.97:0/1107016028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae815df20 con 0x7f2ae80d0660 2024-03-29T04:08:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.658+0000 7f2adf7fe640 1 -- 172.21.15.97:0/1107016028 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2ad8038020 con 0x7f2ae80d0660 2024-03-29T04:08:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.658+0000 7f2adf7fe640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2ac806c560 0x7f2ac806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.658+0000 7f2adffff640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2ac806c560 0x7f2ac806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.658+0000 7f2adf7fe640 1 -- 172.21.15.97:0/1107016028 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2ad80ad150 con 0x7f2ae80d0660 2024-03-29T04:08:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.659+0000 7f2adffff640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2ac806c560 0x7f2ac806ea20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ae815bf80 tx=0x7f2ad0007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:19.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.663+0000 7f2adf7fe640 1 -- 172.21.15.97:0/1107016028 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2ae815df20 con 0x7f2ae80d0660 2024-03-29T04:08:19.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.873+0000 7f2aed190640 1 -- 172.21.15.97:0/1107016028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ae8068640 con 0x7f2ae80d0660 2024-03-29T04:08:19.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.874+0000 7f2adf7fe640 1 -- 172.21.15.97:0/1107016028 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5181 (secure 0 0 0) 0x7f2ad8040070 con 0x7f2ae80d0660 2024-03-29T04:08:19.854 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:19.854 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1127043830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1127043830},{"type":"v1","addr":"172.21.15.154:6834","nonce":1127043830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3767188750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3767188750},{"type":"v1","addr":"172.21.15.97:6835","nonce":3767188750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3651340478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3651340478},{"type":"v1","addr":"172.21.15.97:6837","nonce":3651340478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:07:40.761466+0000","modified":"2024-03-29T04:07:42.771721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3669418265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3669418265},{"type":"v1","addr":"172.21.15.154:6835","nonce":3669418265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-03-29T04:08:19.854 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-03-29T04:08:19.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.876+0000 7f2aed190640 1 -- 172.21.15.97:0/1107016028 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2ac806c560 msgr2=0x7f2ac806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.876+0000 7f2aed190640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2ac806c560 0x7f2ac806ea20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ae815bf80 tx=0x7f2ad0007d50 comp rx=0 tx=0).stop 2024-03-29T04:08:19.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.876+0000 7f2aed190640 1 -- 172.21.15.97:0/1107016028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ae80d0660 msgr2=0x7f2ae815b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.876+0000 7f2aed190640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ae80d0660 0x7f2ae815b4d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8000c00 tx=0x7f2ad80046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.876+0000 7f2aed190640 1 -- 172.21.15.97:0/1107016028 shutdown_connections 2024-03-29T04:08:19.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.876+0000 7f2aed190640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2ac806c560 0x7f2ac806ea20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.876+0000 7f2aed190640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ae8162ab0 0x7f2ae8160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.876+0000 7f2aed190640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ae815e6e0 0x7f2ae81603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.876+0000 7f2aed190640 1 --2- 172.21.15.97:0/1107016028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ae80d0660 0x7f2ae815b4d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.876+0000 7f2aed190640 1 -- 172.21.15.97:0/1107016028 >> 172.21.15.97:0/1107016028 conn(0x7f2ae80b3f10 msgr2=0x7f2ae80b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:19.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.876+0000 7f2aed190640 1 -- 172.21.15.97:0/1107016028 shutdown_connections 2024-03-29T04:08:19.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.876+0000 7f2aed190640 1 -- 172.21.15.97:0/1107016028 wait complete. 2024-03-29T04:08:19.866 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.988+0000 7f1fb780c640 1 Processor -- start 2024-03-29T04:08:19.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.988+0000 7f1fb780c640 1 -- start start 2024-03-29T04:08:19.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.989+0000 7f1fb780c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1fb0070990 0x7f1fb009b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.989+0000 7f1fb780c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb00d05c0 0x7f1fb009aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.989+0000 7f1fb780c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb00d0e10 con 0x7f1fb009af70 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.989+0000 7f1fb780c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f1fb00d10f0 con 0x7f1fb0070990 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.989+0000 7f1fb780c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb00d13d0 con 0x7f1fb00d05c0 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.989+0000 7f1fb5da9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1fb009af70 0x7f1fb015e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35544/0 (socket says 172.21.15.97:35544) 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.989+0000 7f1fb5da9640 1 -- 172.21.15.97:0/1576424234 learned_addr learned my addr 172.21.15.97:0/1576424234 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.990+0000 7f1fb4da7640 1 --2- 172.21.15.97:0/1576424234 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb00d05c0 0x7f1fb009aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.990+0000 7f1fb55a8640 1 --2- 172.21.15.97:0/1576424234 >> v2:172.21.15.154:3300/0 conn(0x7f1fb0070990 0x7f1fb009b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.990+0000 7f1fa7fff640 1 -- 172.21.15.97:0/1576424234 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 596815907 0 0) 0x7f1fb00d0e10 con 0x7f1fb009af70 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.990+0000 7f1fa7fff640 1 -- 172.21.15.97:0/1576424234 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f9c003520 con 0x7f1fb009af70 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.990+0000 7f1fb55a8640 1 -- 172.21.15.97:0/1576424234 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb00d05c0 msgr2=0x7f1fb009aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.990+0000 7f1fb55a8640 1 --2- 172.21.15.97:0/1576424234 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb00d05c0 0x7f1fb009aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.990+0000 7f1fb55a8640 1 -- 172.21.15.97:0/1576424234 >> v1:172.21.15.97:6789/0 conn(0x7f1fb009af70 legacy=0x7f1fb015e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.990+0000 7f1fb55a8640 1 -- 172.21.15.97:0/1576424234 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb00d16b0 con 0x7f1fb0070990 2024-03-29T04:08:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.990+0000 7f1fb4da7640 1 --2- 172.21.15.97:0/1576424234 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb00d05c0 0x7f1fb009aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:19.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.991+0000 7f1fb55a8640 1 --2- 172.21.15.97:0/1576424234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb0070990 0x7f1fb009b930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009d90 tx=0x7f1fac02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86804743858e7985 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:19.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.991+0000 7f1fa7fff640 1 -- 172.21.15.97:0/1576424234 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac039070 con 0x7f1fb0070990 2024-03-29T04:08:19.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.991+0000 7f1fa7fff640 1 -- 172.21.15.97:0/1576424234 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fac02cc50 con 0x7f1fb0070990 2024-03-29T04:08:19.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.991+0000 7f1fa7fff640 1 -- 172.21.15.97:0/1576424234 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac034440 con 0x7f1fb0070990 2024-03-29T04:08:19.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.991+0000 7f1fb780c640 1 -- 172.21.15.97:0/1576424234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb0070990 msgr2=0x7f1fb009b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.991+0000 7f1fb780c640 1 --2- 172.21.15.97:0/1576424234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb0070990 0x7f1fb009b930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009d90 tx=0x7f1fac02bea0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.991+0000 7f1fb780c640 1 -- 172.21.15.97:0/1576424234 shutdown_connections 2024-03-29T04:08:19.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.991+0000 7f1fb780c640 1 --2- 172.21.15.97:0/1576424234 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb00d05c0 0x7f1fb009aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.991+0000 7f1fb780c640 1 --2- 172.21.15.97:0/1576424234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb0070990 0x7f1fb009b930 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.991+0000 7f1fb780c640 1 -- 172.21.15.97:0/1576424234 >> 172.21.15.97:0/1576424234 conn(0x7f1fb00d2430 msgr2=0x7f1fb00d2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:19.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.991+0000 7f1fb780c640 1 -- 172.21.15.97:0/1576424234 shutdown_connections 2024-03-29T04:08:19.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.991+0000 7f1fb780c640 1 -- 172.21.15.97:0/1576424234 wait complete. 2024-03-29T04:08:19.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.992+0000 7f1fb780c640 1 Processor -- start 2024-03-29T04:08:19.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.992+0000 7f1fb780c640 1 -- start start 2024-03-29T04:08:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb780c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fb0070990 0x7f1fb00b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb780c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb009af70 0x7f1fb00c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb780c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb00d05c0 0x7f1fb006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb780c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb00d0ba0 con 0x7f1fb0070990 2024-03-29T04:08:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb780c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb00d0a20 con 0x7f1fb009af70 2024-03-29T04:08:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb780c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb0070d70 con 0x7f1fb00d05c0 2024-03-29T04:08:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb55a8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fb0070990 0x7f1fb00b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb55a8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fb0070990 0x7f1fb00b5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34872/0 (socket says 172.21.15.97:34872) 2024-03-29T04:08:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb55a8640 1 -- 172.21.15.97:0/3107796585 learned_addr learned my addr 172.21.15.97:0/3107796585 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb4da7640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb009af70 0x7f1fb00c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb5da9640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb00d05c0 0x7f1fb006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb55a8640 1 -- 172.21.15.97:0/3107796585 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb00d05c0 msgr2=0x7f1fb006d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb55a8640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb00d05c0 0x7f1fb006d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb55a8640 1 -- 172.21.15.97:0/3107796585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb009af70 msgr2=0x7f1fb00c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.993+0000 7f1fb55a8640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb009af70 0x7f1fb00c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.994+0000 7f1fb55a8640 1 -- 172.21.15.97:0/3107796585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb00c7a00 con 0x7f1fb0070990 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.994+0000 7f1fb4da7640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb009af70 0x7f1fb00c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.994+0000 7f1fb55a8640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fb0070990 0x7f1fb00b5e70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1fac037040 tx=0x7f1fac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.994+0000 7f1fa5ffb640 1 -- 172.21.15.97:0/3107796585 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac039050 con 0x7f1fb0070990 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.994+0000 7f1fa5ffb640 1 -- 172.21.15.97:0/3107796585 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fac045070 con 0x7f1fb0070990 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.994+0000 7f1fb780c640 1 -- 172.21.15.97:0/3107796585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb00c7ce0 con 0x7f1fb0070990 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.994+0000 7f1fa5ffb640 1 -- 172.21.15.97:0/3107796585 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac009450 con 0x7f1fb0070990 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.994+0000 7f1fb780c640 1 -- 172.21.15.97:0/3107796585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb00b63b0 con 0x7f1fb0070990 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.995+0000 7f1fb780c640 1 -- 172.21.15.97:0/3107796585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb0068640 con 0x7f1fb0070990 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.995+0000 7f1fa5ffb640 1 -- 172.21.15.97:0/3107796585 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1fac03e610 con 0x7f1fb0070990 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.996+0000 7f1fa5ffb640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1f9406c530 0x7f1f9406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:19.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.996+0000 7f1fa5ffb640 1 -- 172.21.15.97:0/3107796585 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1fac0ae020 con 0x7f1fb0070990 2024-03-29T04:08:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.996+0000 7f1fb4da7640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1f9406c530 0x7f1f9406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:19.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:19.996+0000 7f1fb4da7640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1f9406c530 0x7f1f9406e9f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0071550 tx=0x7f1fa0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:19.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.000+0000 7f1fa5ffb640 1 -- 172.21.15.97:0/3107796585 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1fac07c700 con 0x7f1fb0070990 2024-03-29T04:08:20.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.174+0000 7f1fb780c640 1 -- 172.21.15.97:0/3107796585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fb00c7eb0 con 0x7f1fb0070990 2024-03-29T04:08:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.175+0000 7f1fa5ffb640 1 -- 172.21.15.97:0/3107796585 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f1fac07c000 con 0x7f1fb0070990 2024-03-29T04:08:20.164 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:20.164 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:07:42.409822+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:07:38.618704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:07:39.724339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.177+0000 7f1fb780c640 1 -- 172.21.15.97:0/3107796585 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1f9406c530 msgr2=0x7f1f9406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.177+0000 7f1fb780c640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1f9406c530 0x7f1f9406e9f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0071550 tx=0x7f1fa0008040 comp rx=0 tx=0).stop 2024-03-29T04:08:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.177+0000 7f1fb780c640 1 -- 172.21.15.97:0/3107796585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fb0070990 msgr2=0x7f1fb00b5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.177+0000 7f1fb780c640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fb0070990 0x7f1fb00b5e70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1fac037040 tx=0x7f1fac004a40 comp rx=0 tx=0).stop 2024-03-29T04:08:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.177+0000 7f1fb780c640 1 -- 172.21.15.97:0/3107796585 shutdown_connections 2024-03-29T04:08:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.177+0000 7f1fb780c640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1f9406c530 0x7f1f9406e9f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.177+0000 7f1fb780c640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1fb00d05c0 0x7f1fb006d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.177+0000 7f1fb780c640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fb009af70 0x7f1fb00c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.177+0000 7f1fb780c640 1 --2- 172.21.15.97:0/3107796585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fb0070990 0x7f1fb00b5e70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.177+0000 7f1fb780c640 1 -- 172.21.15.97:0/3107796585 >> 172.21.15.97:0/3107796585 conn(0x7f1fb00d2430 msgr2=0x7f1fb00b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:20.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.177+0000 7f1fb780c640 1 -- 172.21.15.97:0/3107796585 shutdown_connections 2024-03-29T04:08:20.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.177+0000 7f1fb780c640 1 -- 172.21.15.97:0/3107796585 wait complete. 2024-03-29T04:08:20.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T04:08:20.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.289+0000 7f515a3e8640 1 Processor -- start 2024-03-29T04:08:20.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.289+0000 7f515a3e8640 1 -- start start 2024-03-29T04:08:20.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f515a3e8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f515415fc20 0x7f51540c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:20.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f515a3e8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51540d0f10 con 0x7f51540ccbe0 2024-03-29T04:08:20.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f515a3e8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5154160480 con 0x7f51540c5020 2024-03-29T04:08:20.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f515a3e8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5154161090 con 0x7f515415fc20 2024-03-29T04:08:20.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f5158985640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f51540ccbe0 0x7f51540d0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35558/0 (socket says 172.21.15.97:35558) 2024-03-29T04:08:20.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f5158985640 1 -- 172.21.15.97:0/191182037 learned_addr learned my addr 172.21.15.97:0/191182037 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:20.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f5153fff640 1 --2- 172.21.15.97:0/191182037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f515415fc20 0x7f51540c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:20.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f5152ffd640 1 -- 172.21.15.97:0/191182037 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3775225801 0 0) 0x7f51540d0f10 con 0x7f51540ccbe0 2024-03-29T04:08:20.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f5152ffd640 1 -- 172.21.15.97:0/191182037 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5140003520 con 0x7f51540ccbe0 2024-03-29T04:08:20.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f5152ffd640 1 -- 172.21.15.97:0/191182037 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3357475202 0 0) 0x7f5154160480 con 0x7f51540c5020 2024-03-29T04:08:20.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f5152ffd640 1 -- 172.21.15.97:0/191182037 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51540d0f10 con 0x7f51540c5020 2024-03-29T04:08:20.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f5153fff640 1 -- 172.21.15.97:0/191182037 >> v1:172.21.15.154:6789/0 conn(0x7f51540c5020 legacy=0x7f51540d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:20.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f5153fff640 1 -- 172.21.15.97:0/191182037 >> v1:172.21.15.97:6789/0 conn(0x7f51540ccbe0 legacy=0x7f51540d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:20.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.290+0000 7f5153fff640 1 -- 172.21.15.97:0/191182037 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5154160760 con 0x7f515415fc20 2024-03-29T04:08:20.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.291+0000 7f5153fff640 1 --2- 172.21.15.97:0/191182037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f515415fc20 0x7f51540c6140 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f51440097d0 tx=0x7f514402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5e45af64e9439f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:20.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.291+0000 7f5152ffd640 1 -- 172.21.15.97:0/191182037 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144035020 con 0x7f515415fc20 2024-03-29T04:08:20.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.291+0000 7f5152ffd640 1 -- 172.21.15.97:0/191182037 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5144031800 con 0x7f515415fc20 2024-03-29T04:08:20.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.291+0000 7f5152ffd640 1 -- 172.21.15.97:0/191182037 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144031bd0 con 0x7f515415fc20 2024-03-29T04:08:20.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.292+0000 7f515a3e8640 1 -- 172.21.15.97:0/191182037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f515415fc20 msgr2=0x7f51540c6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:20.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.292+0000 7f515a3e8640 1 --2- 172.21.15.97:0/191182037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f515415fc20 0x7f51540c6140 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f51440097d0 tx=0x7f514402b8a0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.292+0000 7f515a3e8640 1 -- 172.21.15.97:0/191182037 shutdown_connections 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.292+0000 7f515a3e8640 1 --2- 172.21.15.97:0/191182037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f515415fc20 0x7f51540c6140 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.292+0000 7f515a3e8640 1 -- 172.21.15.97:0/191182037 >> 172.21.15.97:0/191182037 conn(0x7f515406b510 msgr2=0x7f515406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.292+0000 7f515a3e8640 1 -- 172.21.15.97:0/191182037 shutdown_connections 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.292+0000 7f515a3e8640 1 -- 172.21.15.97:0/191182037 wait complete. 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.293+0000 7f515a3e8640 1 Processor -- start 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.293+0000 7f515a3e8640 1 -- start start 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f515a3e8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51540c5020 0x7f515414c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f515a3e8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51540ccbe0 0x7f51540c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f515a3e8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f515415fc20 0x7f51540c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f515a3e8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f51541600e0 con 0x7f51540c5020 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f515a3e8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5154160d20 con 0x7f51540ccbe0 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f515a3e8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5154160260 con 0x7f515415fc20 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f5153fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51540c5020 0x7f515414c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:20.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f5153fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51540c5020 0x7f515414c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34878/0 (socket says 172.21.15.97:34878) 2024-03-29T04:08:20.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f5153fff640 1 -- 172.21.15.97:0/3499940185 learned_addr learned my addr 172.21.15.97:0/3499940185 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:20.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f5158985640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f515415fc20 0x7f51540c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:20.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f51537fe640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51540ccbe0 0x7f51540c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:20.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f5153fff640 1 -- 172.21.15.97:0/3499940185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f515415fc20 msgr2=0x7f51540c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:20.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f5153fff640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f515415fc20 0x7f51540c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f5153fff640 1 -- 172.21.15.97:0/3499940185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51540ccbe0 msgr2=0x7f51540c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:20.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f5153fff640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51540ccbe0 0x7f51540c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f5153fff640 1 -- 172.21.15.97:0/3499940185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51540addc0 con 0x7f51540c5020 2024-03-29T04:08:20.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.294+0000 7f5153fff640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51540c5020 0x7f515414c660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5144034040 tx=0x7f51440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:20.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.295+0000 7f5150ff9640 1 -- 172.21.15.97:0/3499940185 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144040070 con 0x7f51540c5020 2024-03-29T04:08:20.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.295+0000 7f5150ff9640 1 -- 172.21.15.97:0/3499940185 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5144031e30 con 0x7f51540c5020 2024-03-29T04:08:20.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.295+0000 7f515a3e8640 1 -- 172.21.15.97:0/3499940185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51540ae0a0 con 0x7f51540c5020 2024-03-29T04:08:20.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.295+0000 7f5150ff9640 1 -- 172.21.15.97:0/3499940185 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144035040 con 0x7f51540c5020 2024-03-29T04:08:20.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.295+0000 7f515a3e8640 1 -- 172.21.15.97:0/3499940185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51540ae660 con 0x7f51540c5020 2024-03-29T04:08:20.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.296+0000 7f515a3e8640 1 -- 172.21.15.97:0/3499940185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5154068640 con 0x7f51540c5020 2024-03-29T04:08:20.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.296+0000 7f5150ff9640 1 -- 172.21.15.97:0/3499940185 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f514404b060 con 0x7f51540c5020 2024-03-29T04:08:20.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.296+0000 7f5150ff9640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f513406c530 0x7f513406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:20.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.296+0000 7f5150ff9640 1 -- 172.21.15.97:0/3499940185 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f51440acea0 con 0x7f51540c5020 2024-03-29T04:08:20.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.296+0000 7f51537fe640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f513406c530 0x7f513406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:20.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.297+0000 7f51537fe640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f513406c530 0x7f513406e9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f513c007fd0 tx=0x7f513c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:20.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.301+0000 7f5150ff9640 1 -- 172.21.15.97:0/3499940185 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f514407af00 con 0x7f51540c5020 2024-03-29T04:08:20.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.513+0000 7f515a3e8640 1 -- 172.21.15.97:0/3499940185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f515415b030 con 0x7f51540c5020 2024-03-29T04:08:20.493 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-03-29T04:08:20.513+0000 7f7c3447e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:08:20.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.514+0000 7f5150ff9640 1 -- 172.21.15.97:0/3499940185 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) ==== 161+0+0 (secure 0 0 0) 0x7f514407f200 con 0x7f51540c5020 2024-03-29T04:08:20.494 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T04:08:20.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.516+0000 7f515a3e8640 1 -- 172.21.15.97:0/3499940185 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f513406c530 msgr2=0x7f513406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:20.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.516+0000 7f515a3e8640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f513406c530 0x7f513406e9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f513c007fd0 tx=0x7f513c009040 comp rx=0 tx=0).stop 2024-03-29T04:08:20.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.516+0000 7f515a3e8640 1 -- 172.21.15.97:0/3499940185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51540c5020 msgr2=0x7f515414c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:20.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.516+0000 7f515a3e8640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51540c5020 0x7f515414c660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5144034040 tx=0x7f51440046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.516+0000 7f515a3e8640 1 -- 172.21.15.97:0/3499940185 shutdown_connections 2024-03-29T04:08:20.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.516+0000 7f515a3e8640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f513406c530 0x7f513406e9f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.516+0000 7f515a3e8640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f515415fc20 0x7f51540c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.516+0000 7f515a3e8640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51540ccbe0 0x7f51540c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.516+0000 7f515a3e8640 1 --2- 172.21.15.97:0/3499940185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51540c5020 0x7f515414c660 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.517+0000 7f515a3e8640 1 -- 172.21.15.97:0/3499940185 >> 172.21.15.97:0/3499940185 conn(0x7f515406b510 msgr2=0x7f51540b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:20.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.517+0000 7f515a3e8640 1 -- 172.21.15.97:0/3499940185 shutdown_connections 2024-03-29T04:08:20.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.517+0000 7f515a3e8640 1 -- 172.21.15.97:0/3499940185 wait complete. 2024-03-29T04:08:20.501 INFO:tasks.ceph.mds.d.smithi154.stderr: -7513> 2024-03-29T04:07:45.195+0000 7f8da22aa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T04:08:20.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T04:08:20.533 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2024-03-29T04:08:20.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.629+0000 7f2787455640 1 Processor -- start 2024-03-29T04:08:20.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.630+0000 7f2787455640 1 -- start start 2024-03-29T04:08:20.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.631+0000 7f2787455640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27800b7140 0x7f27801458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:20.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.631+0000 7f2787455640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2780145e00 0x7f27800b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:20.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.631+0000 7f2787455640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f278006b690 con 0x7f27800cd880 2024-03-29T04:08:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.631+0000 7f2787455640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f278006b970 con 0x7f2780145e00 2024-03-29T04:08:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.631+0000 7f2787455640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f278006bc50 con 0x7f27800b7140 2024-03-29T04:08:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.631+0000 7f27849f0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2780145e00 0x7f27800b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.631+0000 7f27849f0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2780145e00 0x7f27800b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49950/0 (socket says 172.21.15.97:49950) 2024-03-29T04:08:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.631+0000 7f27849f0640 1 -- 172.21.15.97:0/164195899 learned_addr learned my addr 172.21.15.97:0/164195899 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.631+0000 7f27851f1640 1 --2- 172.21.15.97:0/164195899 >> v2:172.21.15.154:3300/0 conn(0x7f27800b7140 0x7f27801458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.631+0000 7f27849f0640 1 -- 172.21.15.97:0/164195899 >> v2:172.21.15.154:3300/0 conn(0x7f27800b7140 msgr2=0x7f27801458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.631+0000 7f27849f0640 1 --2- 172.21.15.97:0/164195899 >> v2:172.21.15.154:3300/0 conn(0x7f27800b7140 0x7f27801458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.631+0000 7f27849f0640 1 -- 172.21.15.97:0/164195899 >> v1:172.21.15.154:6789/0 conn(0x7f27800cd880 legacy=0x7f27800cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.631+0000 7f27849f0640 1 -- 172.21.15.97:0/164195899 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278006bf30 con 0x7f2780145e00 2024-03-29T04:08:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.632+0000 7f27849f0640 1 --2- 172.21.15.97:0/164195899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2780145e00 0x7f27800b2140 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f27800cdce0 tx=0x7f277002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9227eb6c2d04b2ae server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.632+0000 7f2777fff640 1 -- 172.21.15.97:0/164195899 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770035020 con 0x7f2780145e00 2024-03-29T04:08:20.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.632+0000 7f2777fff640 1 -- 172.21.15.97:0/164195899 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2770031870 con 0x7f2780145e00 2024-03-29T04:08:20.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.632+0000 7f2777fff640 1 -- 172.21.15.97:0/164195899 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770031c40 con 0x7f2780145e00 2024-03-29T04:08:20.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.632+0000 7f2787455640 1 -- 172.21.15.97:0/164195899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2780145e00 msgr2=0x7f27800b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:20.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.632+0000 7f2787455640 1 --2- 172.21.15.97:0/164195899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2780145e00 0x7f27800b2140 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f27800cdce0 tx=0x7f277002b910 comp rx=0 tx=0).stop 2024-03-29T04:08:20.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.632+0000 7f2787455640 1 -- 172.21.15.97:0/164195899 shutdown_connections 2024-03-29T04:08:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.632+0000 7f2787455640 1 --2- 172.21.15.97:0/164195899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2780145e00 0x7f27800b2140 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.632+0000 7f2787455640 1 --2- 172.21.15.97:0/164195899 >> v2:172.21.15.154:3300/0 conn(0x7f27800b7140 0x7f27801458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.632+0000 7f2787455640 1 -- 172.21.15.97:0/164195899 >> 172.21.15.97:0/164195899 conn(0x7f278009aa30 msgr2=0x7f27800b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.633+0000 7f2787455640 1 -- 172.21.15.97:0/164195899 shutdown_connections 2024-03-29T04:08:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.633+0000 7f2787455640 1 -- 172.21.15.97:0/164195899 wait complete. 2024-03-29T04:08:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.634+0000 7f2787455640 1 Processor -- start 2024-03-29T04:08:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.634+0000 7f2787455640 1 -- start start 2024-03-29T04:08:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.634+0000 7f2787455640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27800b7140 0x7f278015fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.634+0000 7f2787455640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27800cd880 0x7f278015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.634+0000 7f2787455640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2780145e00 0x7f278015d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.634+0000 7f2787455640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27800b2680 con 0x7f27800cd880 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.634+0000 7f2787455640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f27800b54d0 con 0x7f27800b7140 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.634+0000 7f2787455640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f27800b2800 con 0x7f2780145e00 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.634+0000 7f27849f0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27800cd880 0x7f278015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f27849f0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27800cd880 0x7f278015cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49962/0 (socket says 172.21.15.97:49962) 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f27849f0640 1 -- 172.21.15.97:0/1360143562 learned_addr learned my addr 172.21.15.97:0/1360143562 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f27851f1640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27800b7140 0x7f278015fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f27859f2640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2780145e00 0x7f278015d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f27849f0640 1 -- 172.21.15.97:0/1360143562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2780145e00 msgr2=0x7f278015d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f27849f0640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2780145e00 0x7f278015d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f27849f0640 1 -- 172.21.15.97:0/1360143562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27800b7140 msgr2=0x7f278015fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f27849f0640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27800b7140 0x7f278015fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f27849f0640 1 -- 172.21.15.97:0/1360143562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2780143680 con 0x7f27800cd880 2024-03-29T04:08:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f27849f0640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27800cd880 0x7f278015cc10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2770000c00 tx=0x7f27700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:20.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f2775ffb640 1 -- 172.21.15.97:0/1360143562 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770040070 con 0x7f27800cd880 2024-03-29T04:08:20.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f2775ffb640 1 -- 172.21.15.97:0/1360143562 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2770030d00 con 0x7f27800cd880 2024-03-29T04:08:20.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f2787455640 1 -- 172.21.15.97:0/1360143562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2780143960 con 0x7f27800cd880 2024-03-29T04:08:20.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f2775ffb640 1 -- 172.21.15.97:0/1360143562 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770035040 con 0x7f27800cd880 2024-03-29T04:08:20.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.635+0000 7f2787455640 1 -- 172.21.15.97:0/1360143562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2780143f20 con 0x7f27800cd880 2024-03-29T04:08:20.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.636+0000 7f2787455640 1 -- 172.21.15.97:0/1360143562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278009f610 con 0x7f27800cd880 2024-03-29T04:08:20.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.637+0000 7f2775ffb640 1 -- 172.21.15.97:0/1360143562 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f277004b060 con 0x7f27800cd880 2024-03-29T04:08:20.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.637+0000 7f2775ffb640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f276406c530 0x7f276406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:20.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.637+0000 7f27851f1640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f276406c530 0x7f276406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:20.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.637+0000 7f2775ffb640 1 -- 172.21.15.97:0/1360143562 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f27700ad400 con 0x7f27800cd880 2024-03-29T04:08:20.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.638+0000 7f27851f1640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f276406c530 0x7f276406e9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f277c006fd0 tx=0x7f277c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:20.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.642+0000 7f2775ffb640 1 -- 172.21.15.97:0/1360143562 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f277007b370 con 0x7f27800cd880 2024-03-29T04:08:20.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:20.857+0000 7f2787455640 1 -- 172.21.15.97:0/1360143562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f278006ec80 con 0x7f27800cd880 2024-03-29T04:08:21.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.562+0000 7f2775ffb640 1 -- 172.21.15.97:0/1360143562 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) ==== 104+0+0 (secure 0 0 0) 0x7f277007f670 con 0x7f27800cd880 2024-03-29T04:08:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.565+0000 7f2787455640 1 -- 172.21.15.97:0/1360143562 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f276406c530 msgr2=0x7f276406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.565+0000 7f2787455640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f276406c530 0x7f276406e9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f277c006fd0 tx=0x7f277c008040 comp rx=0 tx=0).stop 2024-03-29T04:08:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.565+0000 7f2787455640 1 -- 172.21.15.97:0/1360143562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27800cd880 msgr2=0x7f278015cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.565+0000 7f2787455640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27800cd880 0x7f278015cc10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2770000c00 tx=0x7f27700046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.565+0000 7f2787455640 1 -- 172.21.15.97:0/1360143562 shutdown_connections 2024-03-29T04:08:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.565+0000 7f2787455640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f276406c530 0x7f276406e9f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.565+0000 7f2787455640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2780145e00 0x7f278015d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.565+0000 7f2787455640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27800cd880 0x7f278015cc10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.565+0000 7f2787455640 1 --2- 172.21.15.97:0/1360143562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27800b7140 0x7f278015fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.565+0000 7f2787455640 1 -- 172.21.15.97:0/1360143562 >> 172.21.15.97:0/1360143562 conn(0x7f278009aa30 msgr2=0x7f27800cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.565+0000 7f2787455640 1 -- 172.21.15.97:0/1360143562 shutdown_connections 2024-03-29T04:08:21.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.565+0000 7f2787455640 1 -- 172.21.15.97:0/1360143562 wait complete. 2024-03-29T04:08:21.558 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T04:08:21.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.686+0000 7f2993aba640 1 Processor -- start 2024-03-29T04:08:21.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.686+0000 7f2993aba640 1 -- start start 2024-03-29T04:08:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.687+0000 7f2993aba640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f298c09b770 0x7f298c0cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.687+0000 7f2993aba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f298c09aa30 0x7f298c09ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.687+0000 7f2993aba640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f298c0ccfc0 con 0x7f298c15fe70 2024-03-29T04:08:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.687+0000 7f2993aba640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f298c09b3c0 con 0x7f298c09b770 2024-03-29T04:08:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.687+0000 7f2993aba640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f298c0cd8b0 con 0x7f298c09aa30 2024-03-29T04:08:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.687+0000 7f2991055640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f298c09b770 0x7f298c0cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.687+0000 7f2991856640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f298c15fe70 0x7f298c0d0e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59084/0 (socket says 172.21.15.97:59084) 2024-03-29T04:08:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.687+0000 7f2991856640 1 -- 172.21.15.97:0/130054335 learned_addr learned my addr 172.21.15.97:0/130054335 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:21.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.687+0000 7f2991055640 1 -- 172.21.15.97:0/130054335 >> v2:172.21.15.154:3300/0 conn(0x7f298c09aa30 msgr2=0x7f298c09ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:21.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.687+0000 7f2991055640 1 --2- 172.21.15.97:0/130054335 >> v2:172.21.15.154:3300/0 conn(0x7f298c09aa30 0x7f298c09ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:21.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.687+0000 7f2991055640 1 -- 172.21.15.97:0/130054335 >> v1:172.21.15.97:6789/0 conn(0x7f298c15fe70 legacy=0x7f298c0d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:21.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.687+0000 7f2990854640 1 -- 172.21.15.97:0/130054335 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2862540171 0 0) 0x7f298c0ccfc0 con 0x7f298c15fe70 2024-03-29T04:08:21.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.687+0000 7f2991055640 1 -- 172.21.15.97:0/130054335 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298c09bcd0 con 0x7f298c09b770 2024-03-29T04:08:21.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.688+0000 7f2991055640 1 --2- 172.21.15.97:0/130054335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298c09b770 0x7f298c0cdd40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f298c06f3d0 tx=0x7f297c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63702ec326853e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:21.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.688+0000 7f2990854640 1 -- 172.21.15.97:0/130054335 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c035020 con 0x7f298c09b770 2024-03-29T04:08:21.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.688+0000 7f2990854640 1 -- 172.21.15.97:0/130054335 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f297c031830 con 0x7f298c09b770 2024-03-29T04:08:21.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.688+0000 7f2990854640 1 -- 172.21.15.97:0/130054335 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c031c00 con 0x7f298c09b770 2024-03-29T04:08:21.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.689+0000 7f2993aba640 1 -- 172.21.15.97:0/130054335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298c09b770 msgr2=0x7f298c0cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:21.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.689+0000 7f2993aba640 1 --2- 172.21.15.97:0/130054335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298c09b770 0x7f298c0cdd40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f298c06f3d0 tx=0x7f297c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:21.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.689+0000 7f2993aba640 1 -- 172.21.15.97:0/130054335 shutdown_connections 2024-03-29T04:08:21.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.689+0000 7f2993aba640 1 --2- 172.21.15.97:0/130054335 >> v2:172.21.15.154:3300/0 conn(0x7f298c09aa30 0x7f298c09ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:21.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.689+0000 7f2993aba640 1 --2- 172.21.15.97:0/130054335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298c09b770 0x7f298c0cdd40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:21.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.689+0000 7f2993aba640 1 -- 172.21.15.97:0/130054335 >> 172.21.15.97:0/130054335 conn(0x7f298c0b3f10 msgr2=0x7f298c0b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:21.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.689+0000 7f2993aba640 1 -- 172.21.15.97:0/130054335 shutdown_connections 2024-03-29T04:08:21.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.689+0000 7f2993aba640 1 -- 172.21.15.97:0/130054335 wait complete. 2024-03-29T04:08:21.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.690+0000 7f2993aba640 1 Processor -- start 2024-03-29T04:08:21.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.690+0000 7f2993aba640 1 -- start start 2024-03-29T04:08:21.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2993aba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f298c09aa30 0x7f298c0c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:21.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2993aba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298c09b770 0x7f298c159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:21.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2993aba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f298c15fe70 0x7f298c0c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2993aba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f298c0ce280 con 0x7f298c09b770 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2993aba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f298c0cd540 con 0x7f298c15fe70 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2993aba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f298c158c30 con 0x7f298c09aa30 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2991055640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298c09b770 0x7f298c159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2991055640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298c09b770 0x7f298c159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49982/0 (socket says 172.21.15.97:49982) 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2991055640 1 -- 172.21.15.97:0/4176706370 learned_addr learned my addr 172.21.15.97:0/4176706370 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2992057640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f298c15fe70 0x7f298c0c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2991856640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f298c09aa30 0x7f298c0c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2991055640 1 -- 172.21.15.97:0/4176706370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f298c09aa30 msgr2=0x7f298c0c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2991055640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f298c09aa30 0x7f298c0c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2991055640 1 -- 172.21.15.97:0/4176706370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f298c15fe70 msgr2=0x7f298c0c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2991055640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f298c15fe70 0x7f298c0c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2991055640 1 -- 172.21.15.97:0/4176706370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298c158ed0 con 0x7f298c09b770 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2992057640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f298c15fe70 0x7f298c0c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:21.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2991055640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298c09b770 0x7f298c159f70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f297c009890 tx=0x7f297c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:21.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f29827fc640 1 -- 172.21.15.97:0/4176706370 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c03ba80 con 0x7f298c09b770 2024-03-29T04:08:21.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f29827fc640 1 -- 172.21.15.97:0/4176706370 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f297c03bc40 con 0x7f298c09b770 2024-03-29T04:08:21.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2993aba640 1 -- 172.21.15.97:0/4176706370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298c1591b0 con 0x7f298c09b770 2024-03-29T04:08:21.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f29827fc640 1 -- 172.21.15.97:0/4176706370 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c0453f0 con 0x7f298c09b770 2024-03-29T04:08:21.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.691+0000 7f2993aba640 1 -- 172.21.15.97:0/4176706370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f298c159770 con 0x7f298c09b770 2024-03-29T04:08:21.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.692+0000 7f2993aba640 1 -- 172.21.15.97:0/4176706370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f298c068640 con 0x7f298c09b770 2024-03-29T04:08:21.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.693+0000 7f29827fc640 1 -- 172.21.15.97:0/4176706370 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f297c038020 con 0x7f298c09b770 2024-03-29T04:08:21.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.693+0000 7f29827fc640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f297006c530 0x7f297006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:21.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.693+0000 7f29827fc640 1 -- 172.21.15.97:0/4176706370 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5911+0+0 (secure 0 0 0) 0x7f297c0ad640 con 0x7f298c09b770 2024-03-29T04:08:21.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.693+0000 7f2991856640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f297006c530 0x7f297006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:21.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.694+0000 7f2991856640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f297006c530 0x7f297006e9f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2988000f60 tx=0x7f2988007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:21.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.698+0000 7f29827fc640 1 -- 172.21.15.97:0/4176706370 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f297c040070 con 0x7f298c09b770 2024-03-29T04:08:21.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:21.873+0000 7f2993aba640 1 -- 172.21.15.97:0/4176706370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f298c0cec10 con 0x7f298c09b770 2024-03-29T04:08:22.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.564+0000 7f29827fc640 1 -- 172.21.15.97:0/4176706370 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7f297c07bb90 con 0x7f298c09b770 2024-03-29T04:08:22.543 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-03-29T04:08:22.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.566+0000 7f2993aba640 1 -- 172.21.15.97:0/4176706370 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f297006c530 msgr2=0x7f297006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:22.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.566+0000 7f2993aba640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f297006c530 0x7f297006e9f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2988000f60 tx=0x7f2988007c00 comp rx=0 tx=0).stop 2024-03-29T04:08:22.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.567+0000 7f2993aba640 1 -- 172.21.15.97:0/4176706370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298c09b770 msgr2=0x7f298c159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:22.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.567+0000 7f2993aba640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298c09b770 0x7f298c159f70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f297c009890 tx=0x7f297c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:22.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.567+0000 7f2993aba640 1 -- 172.21.15.97:0/4176706370 shutdown_connections 2024-03-29T04:08:22.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.567+0000 7f2993aba640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f297006c530 0x7f297006e9f0 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2988000f60 tx=0x7f2988007c00 comp rx=0 tx=0).stop 2024-03-29T04:08:22.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.567+0000 7f2993aba640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f298c15fe70 0x7f298c0c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:22.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.567+0000 7f2993aba640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298c09b770 0x7f298c159f70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:22.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.567+0000 7f2993aba640 1 --2- 172.21.15.97:0/4176706370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f298c09aa30 0x7f298c0c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:22.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.567+0000 7f2993aba640 1 -- 172.21.15.97:0/4176706370 >> 172.21.15.97:0/4176706370 conn(0x7f298c0b3f10 msgr2=0x7f298c0afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:22.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.568+0000 7f2993aba640 1 -- 172.21.15.97:0/4176706370 shutdown_connections 2024-03-29T04:08:22.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.568+0000 7f2993aba640 1 -- 172.21.15.97:0/4176706370 wait complete. 2024-03-29T04:08:22.565 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T04:08:22.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.692+0000 7f0fc7ce4640 1 Processor -- start 2024-03-29T04:08:22.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.693+0000 7f0fc7ce4640 1 -- start start 2024-03-29T04:08:22.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.693+0000 7f0fc7ce4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0fc00d4f20 0x7f0fc0162ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.693+0000 7f0fc7ce4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0fc015eb60 0x7f0fc00d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.693+0000 7f0fc7ce4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fc00b37c0 con 0x7f0fc015e780 2024-03-29T04:08:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.693+0000 7f0fc7ce4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f0fc0163430 con 0x7f0fc00d4f20 2024-03-29T04:08:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.693+0000 7f0fc7ce4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f0fc00cfde0 con 0x7f0fc015eb60 2024-03-29T04:08:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.693+0000 7f0fc527f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0fc00d4f20 0x7f0fc0162ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.694+0000 7f0fc5a80640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0fc015e780 0x7f0fc00b1170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59100/0 (socket says 172.21.15.97:59100) 2024-03-29T04:08:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.694+0000 7f0fc5a80640 1 -- 172.21.15.97:0/2771448130 learned_addr learned my addr 172.21.15.97:0/2771448130 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.694+0000 7f0fc527f640 1 -- 172.21.15.97:0/2771448130 >> v2:172.21.15.154:3300/0 conn(0x7f0fc015eb60 msgr2=0x7f0fc00d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.694+0000 7f0fc527f640 1 --2- 172.21.15.97:0/2771448130 >> v2:172.21.15.154:3300/0 conn(0x7f0fc015eb60 0x7f0fc00d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.694+0000 7f0fc527f640 1 -- 172.21.15.97:0/2771448130 >> v1:172.21.15.97:6789/0 conn(0x7f0fc015e780 legacy=0x7f0fc00b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.694+0000 7f0fc4a7e640 1 -- 172.21.15.97:0/2771448130 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1740389110 0 0) 0x7f0fc00b37c0 con 0x7f0fc015e780 2024-03-29T04:08:22.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.694+0000 7f0fc527f640 1 -- 172.21.15.97:0/2771448130 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc00d53e0 con 0x7f0fc00d4f20 2024-03-29T04:08:22.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.695+0000 7f0fc527f640 1 --2- 172.21.15.97:0/2771448130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc00d4f20 0x7f0fc0162ef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0fc008c9c0 tx=0x7f0fb002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17805fe37c578a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:22.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.695+0000 7f0fc4a7e640 1 -- 172.21.15.97:0/2771448130 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0035020 con 0x7f0fc00d4f20 2024-03-29T04:08:22.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.695+0000 7f0fc4a7e640 1 -- 172.21.15.97:0/2771448130 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb0031830 con 0x7f0fc00d4f20 2024-03-29T04:08:22.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.695+0000 7f0fc4a7e640 1 -- 172.21.15.97:0/2771448130 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0031c00 con 0x7f0fc00d4f20 2024-03-29T04:08:22.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.695+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/2771448130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc00d4f20 msgr2=0x7f0fc0162ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:22.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.695+0000 7f0fc7ce4640 1 --2- 172.21.15.97:0/2771448130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc00d4f20 0x7f0fc0162ef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0fc008c9c0 tx=0x7f0fb002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:22.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.695+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/2771448130 shutdown_connections 2024-03-29T04:08:22.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.695+0000 7f0fc7ce4640 1 --2- 172.21.15.97:0/2771448130 >> v2:172.21.15.154:3300/0 conn(0x7f0fc015eb60 0x7f0fc00d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:22.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.695+0000 7f0fc7ce4640 1 --2- 172.21.15.97:0/2771448130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc00d4f20 0x7f0fc0162ef0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:22.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.695+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/2771448130 >> 172.21.15.97:0/2771448130 conn(0x7f0fc008bfc0 msgr2=0x7f0fc00d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:22.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.695+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/2771448130 shutdown_connections 2024-03-29T04:08:22.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.695+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/2771448130 wait complete. 2024-03-29T04:08:22.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.696+0000 7f0fc7ce4640 1 Processor -- start 2024-03-29T04:08:22.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.696+0000 7f0fc7ce4640 1 -- start start 2024-03-29T04:08:22.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.697+0000 7f0fc7ce4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc00d4f20 0x7f0fc01f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:22.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.697+0000 7f0fc7ce4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0fc015e780 0x7f0fc00c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:22.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.697+0000 7f0fc7ce4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0fc015eb60 0x7f0fc00c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:22.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.697+0000 7f0fc7ce4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc00cf440 con 0x7f0fc00d4f20 2024-03-29T04:08:22.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.697+0000 7f0fc7ce4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc00cfa70 con 0x7f0fc015eb60 2024-03-29T04:08:22.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.697+0000 7f0fc7ce4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc00b1880 con 0x7f0fc015e780 2024-03-29T04:08:22.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.697+0000 7f0fc5a80640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc00d4f20 0x7f0fc01f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:22.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.697+0000 7f0fc5a80640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc00d4f20 0x7f0fc01f3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50010/0 (socket says 172.21.15.97:50010) 2024-03-29T04:08:22.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.697+0000 7f0fc5a80640 1 -- 172.21.15.97:0/1989074689 learned_addr learned my addr 172.21.15.97:0/1989074689 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:22.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.697+0000 7f0fc6281640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0fc015eb60 0x7f0fc00c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:22.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.697+0000 7f0fc527f640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0fc015e780 0x7f0fc00c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:22.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.697+0000 7f0fc5a80640 1 -- 172.21.15.97:0/1989074689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0fc015e780 msgr2=0x7f0fc00c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:22.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.697+0000 7f0fc5a80640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0fc015e780 0x7f0fc00c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:22.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.698+0000 7f0fc5a80640 1 -- 172.21.15.97:0/1989074689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0fc015eb60 msgr2=0x7f0fc00c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.698+0000 7f0fc5a80640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0fc015eb60 0x7f0fc00c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.698+0000 7f0fc5a80640 1 -- 172.21.15.97:0/1989074689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc00c8a60 con 0x7f0fc00d4f20 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.698+0000 7f0fc6281640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0fc015eb60 0x7f0fc00c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.698+0000 7f0fc527f640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0fc015e780 0x7f0fc00c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.698+0000 7f0fc5a80640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc00d4f20 0x7f0fc01f3ac0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00c4d0 tx=0x7f0fbc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.698+0000 7f0fb67fc640 1 -- 172.21.15.97:0/1989074689 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc018070 con 0x7f0fc00d4f20 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.698+0000 7f0fb67fc640 1 -- 172.21.15.97:0/1989074689 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fbc007500 con 0x7f0fc00d4f20 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.698+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/1989074689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc00c8d40 con 0x7f0fc00d4f20 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.698+0000 7f0fb67fc640 1 -- 172.21.15.97:0/1989074689 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc004a20 con 0x7f0fc00d4f20 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.698+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/1989074689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc00c9300 con 0x7f0fc00d4f20 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.699+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/1989074689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc0068640 con 0x7f0fc00d4f20 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.699+0000 7f0fb67fc640 1 -- 172.21.15.97:0/1989074689 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0fbc01b070 con 0x7f0fc00d4f20 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.700+0000 7f0fb67fc640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0fa406c530 0x7f0fa406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.700+0000 7f0fb67fc640 1 -- 172.21.15.97:0/1989074689 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5473+0+0 (secure 0 0 0) 0x7f0fbc08a8d0 con 0x7f0fc00d4f20 2024-03-29T04:08:22.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.700+0000 7f0fc527f640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0fa406c530 0x7f0fa406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:22.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.700+0000 7f0fc527f640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0fa406c530 0x7f0fa406e9f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0036000 tx=0x7f0fb003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:22.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.704+0000 7f0fb67fc640 1 -- 172.21.15.97:0/1989074689 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0fbc059060 con 0x7f0fc00d4f20 2024-03-29T04:08:22.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:22.878+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/1989074689 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0fc00d1e10 con 0x7f0fc00d4f20 2024-03-29T04:08:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.568+0000 7f0fb67fc640 1 -- 172.21.15.97:0/1989074689 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7f0fbc058960 con 0x7f0fc00d4f20 2024-03-29T04:08:23.548 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-03-29T04:08:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.571+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/1989074689 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0fa406c530 msgr2=0x7f0fa406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.571+0000 7f0fc7ce4640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0fa406c530 0x7f0fa406e9f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0036000 tx=0x7f0fb003c040 comp rx=0 tx=0).stop 2024-03-29T04:08:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.571+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/1989074689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc00d4f20 msgr2=0x7f0fc01f3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.571+0000 7f0fc7ce4640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc00d4f20 0x7f0fc01f3ac0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00c4d0 tx=0x7f0fbc00c990 comp rx=0 tx=0).stop 2024-03-29T04:08:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.571+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/1989074689 shutdown_connections 2024-03-29T04:08:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.571+0000 7f0fc7ce4640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0fa406c530 0x7f0fa406e9f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.571+0000 7f0fc7ce4640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0fc015eb60 0x7f0fc00c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.571+0000 7f0fc7ce4640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0fc015e780 0x7f0fc00c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.571+0000 7f0fc7ce4640 1 --2- 172.21.15.97:0/1989074689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fc00d4f20 0x7f0fc01f3ac0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.571+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/1989074689 >> 172.21.15.97:0/1989074689 conn(0x7f0fc008bfc0 msgr2=0x7f0fc006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.571+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/1989074689 shutdown_connections 2024-03-29T04:08:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.571+0000 7f0fc7ce4640 1 -- 172.21.15.97:0/1989074689 wait complete. 2024-03-29T04:08:23.564 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T04:08:23.564 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-29T04:08:23.564 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T04:08:23.564 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:08:23.565 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T04:08:23.565 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-29T04:08:23.565 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T04:08:23.565 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:08:23.565 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T04:08:23.565 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-29T04:08:23.566 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T04:08:23.566 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:08:23.566 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T04:08:23.566 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-29T04:08:23.566 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T04:08:23.566 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:08:23.574 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T04:08:23.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T04:08:23.575 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T04:08:23.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T04:08:23.579 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T04:08:23.579 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T04:08:23.580 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T04:08:23.580 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T04:08:23.617 INFO:tasks.ceph.mds.a:Started 2024-03-29T04:08:23.617 DEBUG:teuthology.parallel:result is None 2024-03-29T04:08:23.618 INFO:tasks.ceph.mds.c:Started 2024-03-29T04:08:23.618 DEBUG:teuthology.parallel:result is None 2024-03-29T04:08:23.623 INFO:tasks.ceph.mds.b:Started 2024-03-29T04:08:23.624 DEBUG:teuthology.parallel:result is None 2024-03-29T04:08:23.625 INFO:tasks.ceph.mds.d:Started 2024-03-29T04:08:23.625 DEBUG:teuthology.parallel:result is None 2024-03-29T04:08:23.625 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-29T04:08:23.695 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-03-29T04:08:23.695 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-03-29T04:08:23.706 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2024-03-29T04:08:23.707 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2024-03-29T04:08:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.753+0000 7fe73ac38640 1 Processor -- start 2024-03-29T04:08:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.754+0000 7fe73ac38640 1 -- start start 2024-03-29T04:08:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.754+0000 7fe73ac38640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe73415eb60 0x7fe7340cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.754+0000 7fe73ac38640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7340cfe70 con 0x7fe7340ba000 2024-03-29T04:08:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.754+0000 7fe73ac38640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7340b7f50 con 0x7fe73415e780 2024-03-29T04:08:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.754+0000 7fe73ac38640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe7340b8230 con 0x7fe73415eb60 2024-03-29T04:08:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.754+0000 7fe733fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe73415eb60 0x7fe7340cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.754+0000 7fe7391d5640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe7340ba000 0x7fe7340d2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59106/0 (socket says 172.21.15.97:59106) 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.754+0000 7fe733fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe73415eb60 0x7fe7340cf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50050/0 (socket says 172.21.15.97:50050) 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.754+0000 7fe7391d5640 1 -- 172.21.15.97:0/1254330733 learned_addr learned my addr 172.21.15.97:0/1254330733 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.755+0000 7fe733fff640 1 -- 172.21.15.97:0/1254330733 >> v1:172.21.15.154:6789/0 conn(0x7fe73415e780 legacy=0x7fe7340cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.755+0000 7fe7337fe640 1 -- 172.21.15.97:0/1254330733 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2527562453 0 0) 0x7fe7340cfe70 con 0x7fe7340ba000 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.755+0000 7fe733fff640 1 -- 172.21.15.97:0/1254330733 >> v1:172.21.15.97:6789/0 conn(0x7fe7340ba000 legacy=0x7fe7340d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.755+0000 7fe733fff640 1 -- 172.21.15.97:0/1254330733 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7340b8510 con 0x7fe73415eb60 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.756+0000 7fe733fff640 1 --2- 172.21.15.97:0/1254330733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73415eb60 0x7fe7340cf2d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0097a0 tx=0x7fe71c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43174926cf2b240b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.756+0000 7fe7337fe640 1 -- 172.21.15.97:0/1254330733 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c034070 con 0x7fe73415eb60 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.756+0000 7fe7337fe640 1 -- 172.21.15.97:0/1254330733 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe71c030610 con 0x7fe73415eb60 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.756+0000 7fe7337fe640 1 -- 172.21.15.97:0/1254330733 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c0309e0 con 0x7fe73415eb60 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.756+0000 7fe73ac38640 1 -- 172.21.15.97:0/1254330733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73415eb60 msgr2=0x7fe7340cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.756+0000 7fe73ac38640 1 --2- 172.21.15.97:0/1254330733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73415eb60 0x7fe7340cf2d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0097a0 tx=0x7fe71c02b960 comp rx=0 tx=0).stop 2024-03-29T04:08:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.756+0000 7fe73ac38640 1 -- 172.21.15.97:0/1254330733 shutdown_connections 2024-03-29T04:08:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.756+0000 7fe73ac38640 1 --2- 172.21.15.97:0/1254330733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73415eb60 0x7fe7340cf2d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.756+0000 7fe73ac38640 1 -- 172.21.15.97:0/1254330733 >> 172.21.15.97:0/1254330733 conn(0x7fe73408bfc0 msgr2=0x7fe7340d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.756+0000 7fe73ac38640 1 -- 172.21.15.97:0/1254330733 shutdown_connections 2024-03-29T04:08:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.756+0000 7fe73ac38640 1 -- 172.21.15.97:0/1254330733 wait complete. 2024-03-29T04:08:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.757+0000 7fe73ac38640 1 Processor -- start 2024-03-29T04:08:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.758+0000 7fe73ac38640 1 -- start start 2024-03-29T04:08:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.758+0000 7fe73ac38640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7340ba000 0x7fe7340ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.758+0000 7fe73ac38640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe73415e780 0x7fe734161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.758+0000 7fe73ac38640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7340cad50 0x7fe734161660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.758+0000 7fe73ac38640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe7340cf810 con 0x7fe7340cad50 2024-03-29T04:08:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.758+0000 7fe73ac38640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe7340b8ad0 con 0x7fe7340ba000 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.758+0000 7fe73ac38640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe7340cf990 con 0x7fe73415e780 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.758+0000 7fe7391d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7340cad50 0x7fe734161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe7391d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7340cad50 0x7fe734161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50056/0 (socket says 172.21.15.97:50056) 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe7391d5640 1 -- 172.21.15.97:0/1867388616 learned_addr learned my addr 172.21.15.97:0/1867388616 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe7389d4640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7340ba000 0x7fe7340ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe733fff640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe73415e780 0x7fe734161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe7391d5640 1 -- 172.21.15.97:0/1867388616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe73415e780 msgr2=0x7fe734161120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe7391d5640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe73415e780 0x7fe734161120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe7391d5640 1 -- 172.21.15.97:0/1867388616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7340ba000 msgr2=0x7fe7340ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe7391d5640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7340ba000 0x7fe7340ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe7391d5640 1 -- 172.21.15.97:0/1867388616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7340b61c0 con 0x7fe7340cad50 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe7391d5640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7340cad50 0x7fe734161660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00c7a0 tx=0x7fe72c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe7317fa640 1 -- 172.21.15.97:0/1867388616 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c018070 con 0x7fe7340cad50 2024-03-29T04:08:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe7317fa640 1 -- 172.21.15.97:0/1867388616 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe72c00a640 con 0x7fe7340cad50 2024-03-29T04:08:23.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe7317fa640 1 -- 172.21.15.97:0/1867388616 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c00aab0 con 0x7fe7340cad50 2024-03-29T04:08:23.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe73ac38640 1 -- 172.21.15.97:0/1867388616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73414e340 con 0x7fe7340cad50 2024-03-29T04:08:23.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.759+0000 7fe73ac38640 1 -- 172.21.15.97:0/1867388616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73414e900 con 0x7fe7340cad50 2024-03-29T04:08:23.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.760+0000 7fe73ac38640 1 -- 172.21.15.97:0/1867388616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe73414ef10 con 0x7fe7340cad50 2024-03-29T04:08:23.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.761+0000 7fe7317fa640 1 -- 172.21.15.97:0/1867388616 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe72c01b070 con 0x7fe7340cad50 2024-03-29T04:08:23.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.761+0000 7fe7317fa640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe71406c530 0x7fe71406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:23.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.762+0000 7fe7317fa640 1 -- 172.21.15.97:0/1867388616 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7fe72c08aa10 con 0x7fe7340cad50 2024-03-29T04:08:23.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.762+0000 7fe7389d4640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe71406c530 0x7fe71406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:23.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.762+0000 7fe7389d4640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe71406c530 0x7fe71406e9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe724000f60 tx=0x7fe724009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:23.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.766+0000 7fe7317fa640 1 -- 172.21.15.97:0/1867388616 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe7340b61c0 con 0x7fe7340cad50 2024-03-29T04:08:23.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:23.951+0000 7fe73ac38640 1 -- 172.21.15.97:0/1867388616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe734068640 con 0x7fe7340cad50 2024-03-29T04:08:24.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.582+0000 7fe7317fa640 1 -- 172.21.15.97:0/1867388616 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7fe72c058c40 con 0x7fe7340cad50 2024-03-29T04:08:24.561 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-03-29T04:08:24.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.584+0000 7fe73ac38640 1 -- 172.21.15.97:0/1867388616 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe71406c530 msgr2=0x7fe71406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:24.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.584+0000 7fe73ac38640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe71406c530 0x7fe71406e9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe724000f60 tx=0x7fe724009040 comp rx=0 tx=0).stop 2024-03-29T04:08:24.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.584+0000 7fe73ac38640 1 -- 172.21.15.97:0/1867388616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7340cad50 msgr2=0x7fe734161660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:24.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.584+0000 7fe73ac38640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7340cad50 0x7fe734161660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00c7a0 tx=0x7fe72c00cc60 comp rx=0 tx=0).stop 2024-03-29T04:08:24.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.584+0000 7fe73ac38640 1 -- 172.21.15.97:0/1867388616 shutdown_connections 2024-03-29T04:08:24.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.584+0000 7fe73ac38640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe71406c530 0x7fe71406e9f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.584+0000 7fe73ac38640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7340cad50 0x7fe734161660 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.584+0000 7fe73ac38640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe73415e780 0x7fe734161120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.584+0000 7fe73ac38640 1 --2- 172.21.15.97:0/1867388616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7340ba000 0x7fe7340ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.584+0000 7fe73ac38640 1 -- 172.21.15.97:0/1867388616 >> 172.21.15.97:0/1867388616 conn(0x7fe73408bfc0 msgr2=0x7fe7340b8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:24.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.584+0000 7fe73ac38640 1 -- 172.21.15.97:0/1867388616 shutdown_connections 2024-03-29T04:08:24.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.584+0000 7fe73ac38640 1 -- 172.21.15.97:0/1867388616 wait complete. 2024-03-29T04:08:24.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-29T04:08:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.697+0000 7f5b34582640 1 Processor -- start 2024-03-29T04:08:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.697+0000 7f5b34582640 1 -- start start 2024-03-29T04:08:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.698+0000 7f5b34582640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b2c15e9b0 0x7f5b2c0d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.698+0000 7f5b34582640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b2c0d4f30 0x7f5b2c162f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.698+0000 7f5b34582640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b2c0d54f0 con 0x7f5b2c0d2bf0 2024-03-29T04:08:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.698+0000 7f5b34582640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f5b2c1634b0 con 0x7f5b2c0d4f30 2024-03-29T04:08:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.698+0000 7f5b34582640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2c0cfcf0 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.698+0000 7f5b31b1d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b2c0d4f30 0x7f5b2c162f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.698+0000 7f5b31b1d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b2c0d4f30 0x7f5b2c162f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50068/0 (socket says 172.21.15.97:50068) 2024-03-29T04:08:24.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.698+0000 7f5b31b1d640 1 -- 172.21.15.97:0/1139932582 learned_addr learned my addr 172.21.15.97:0/1139932582 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.698+0000 7f5b31b1d640 1 -- 172.21.15.97:0/1139932582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b2c15e9b0 msgr2=0x7f5b2c0d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.698+0000 7f5b31b1d640 1 --2- 172.21.15.97:0/1139932582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b2c15e9b0 0x7f5b2c0d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.698+0000 7f5b31b1d640 1 -- 172.21.15.97:0/1139932582 >> v1:172.21.15.154:6789/0 conn(0x7f5b2c0d2bf0 legacy=0x7f5b2c06c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.698+0000 7f5b31b1d640 1 -- 172.21.15.97:0/1139932582 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c0cf3c0 con 0x7f5b2c0d4f30 2024-03-29T04:08:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.699+0000 7f5b31b1d640 1 --2- 172.21.15.97:0/1139932582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b2c0d4f30 0x7f5b2c162f70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c009870 tx=0x7f5b1c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a096c4db9275009 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.699+0000 7f5b3131c640 1 -- 172.21.15.97:0/1139932582 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c035020 con 0x7f5b2c0d4f30 2024-03-29T04:08:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.699+0000 7f5b3131c640 1 -- 172.21.15.97:0/1139932582 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b1c031640 con 0x7f5b2c0d4f30 2024-03-29T04:08:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.699+0000 7f5b3131c640 1 -- 172.21.15.97:0/1139932582 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c02d040 con 0x7f5b2c0d4f30 2024-03-29T04:08:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.699+0000 7f5b34582640 1 -- 172.21.15.97:0/1139932582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b2c0d4f30 msgr2=0x7f5b2c162f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.699+0000 7f5b34582640 1 --2- 172.21.15.97:0/1139932582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b2c0d4f30 0x7f5b2c162f70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c009870 tx=0x7f5b1c02b8f0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.699+0000 7f5b34582640 1 -- 172.21.15.97:0/1139932582 shutdown_connections 2024-03-29T04:08:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.699+0000 7f5b34582640 1 --2- 172.21.15.97:0/1139932582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b2c0d4f30 0x7f5b2c162f70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.699+0000 7f5b34582640 1 --2- 172.21.15.97:0/1139932582 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b2c15e9b0 0x7f5b2c0d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.699+0000 7f5b34582640 1 -- 172.21.15.97:0/1139932582 >> 172.21.15.97:0/1139932582 conn(0x7f5b2c08bfc0 msgr2=0x7f5b2c0d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:24.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.700+0000 7f5b34582640 1 -- 172.21.15.97:0/1139932582 shutdown_connections 2024-03-29T04:08:24.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.700+0000 7f5b34582640 1 -- 172.21.15.97:0/1139932582 wait complete. 2024-03-29T04:08:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.701+0000 7f5b34582640 1 Processor -- start 2024-03-29T04:08:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.701+0000 7f5b34582640 1 -- start start 2024-03-29T04:08:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.701+0000 7f5b34582640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b2c0d2bf0 0x7f5b2c15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.701+0000 7f5b34582640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b2c0d4f30 0x7f5b2c0c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.701+0000 7f5b34582640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b2c15e9b0 0x7f5b2c0c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.701+0000 7f5b34582640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c15ed90 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.701+0000 7f5b34582640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c0cf980 con 0x7f5b2c0d2bf0 2024-03-29T04:08:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.701+0000 7f5b34582640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2c0d3080 con 0x7f5b2c0d4f30 2024-03-29T04:08:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.701+0000 7f5b32b1f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b2c15e9b0 0x7f5b2c0c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.701+0000 7f5b32b1f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b2c15e9b0 0x7f5b2c0c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50076/0 (socket says 172.21.15.97:50076) 2024-03-29T04:08:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.701+0000 7f5b32b1f640 1 -- 172.21.15.97:0/2988256767 learned_addr learned my addr 172.21.15.97:0/2988256767 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:24.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.702+0000 7f5b31b1d640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b2c0d4f30 0x7f5b2c0c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:24.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.702+0000 7f5b3231e640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b2c0d2bf0 0x7f5b2c15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:24.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.702+0000 7f5b32b1f640 1 -- 172.21.15.97:0/2988256767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b2c0d4f30 msgr2=0x7f5b2c0c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:24.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.702+0000 7f5b32b1f640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b2c0d4f30 0x7f5b2c0c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.702+0000 7f5b32b1f640 1 -- 172.21.15.97:0/2988256767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b2c0d2bf0 msgr2=0x7f5b2c15d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:24.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.702+0000 7f5b32b1f640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b2c0d2bf0 0x7f5b2c15d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.702+0000 7f5b32b1f640 1 -- 172.21.15.97:0/2988256767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c0cb880 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.702+0000 7f5b32b1f640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b2c15e9b0 0x7f5b2c0c99a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400c780 tx=0x7f5b2400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:24.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.702+0000 7f5b22ffd640 1 -- 172.21.15.97:0/2988256767 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24004b60 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.702+0000 7f5b22ffd640 1 -- 172.21.15.97:0/2988256767 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b24004d00 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.702+0000 7f5b34582640 1 -- 172.21.15.97:0/2988256767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2c0cbb60 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.702+0000 7f5b22ffd640 1 -- 172.21.15.97:0/2988256767 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2400aa90 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.702+0000 7f5b34582640 1 -- 172.21.15.97:0/2988256767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2c0cc120 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.703+0000 7f5b34582640 1 -- 172.21.15.97:0/2988256767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b2c068640 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.704+0000 7f5b22ffd640 1 -- 172.21.15.97:0/2988256767 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5b2400ac30 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.704+0000 7f5b22ffd640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5b1006c530 0x7f5b1006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:24.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.704+0000 7f5b22ffd640 1 -- 172.21.15.97:0/2988256767 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f5b2408b740 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.704+0000 7f5b3231e640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5b1006c530 0x7f5b1006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:24.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.704+0000 7f5b3231e640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5b1006c530 0x7f5b1006e9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800b810 tx=0x7f5b28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:24.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.709+0000 7f5b22ffd640 1 -- 172.21.15.97:0/2988256767 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5b24020070 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.949+0000 7f5b34582640 1 -- 172.21.15.97:0/2988256767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5b2c0d29d0 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.949+0000 7f5b22ffd640 1 -- 172.21.15.97:0/2988256767 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f5b2405a020 con 0x7f5b2c15e9b0 2024-03-29T04:08:24.929 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:24.929 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:08:24.929 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-03-29T04:08:24.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.929 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-03-29T04:08:24.929 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBSPgZmEj/dKRAAytsfrGa+X1yjseBjTlCkJw==", 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBSPgZmL0g2NhAA28cGRN0Fd4nFlOWU7PNtXg==", 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-03-29T04:08:24.930 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBSPgZm2aQ0MRAAHHqNIpIdDOQscqREfWR5Qw==", 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.931 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBTPgZmaoAFAhAAeDOfrEhIg+cSNEiguyksXA==", 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBWPgZmMa1kIxAAtEFfOrYEUQYoHTAlyX1guw==", 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.932 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBZPgZmNrSeIhAAPAPvi9LSGTMEMTAFcsCqOg==", 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBcPgZmp3D+IBAAd/jYMEC4jy0Kq7cPERtTvg==", 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.933 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBfPgZmC7ngHxAAtLZmZoRdOw5GzLtgr5qjwQ==", 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.934 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBlPgZm6arzJhAAV8CWcM4jEJnd4AYUYrwIEw==", 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBoPgZmtv0SJRAAethE1tyjHhFfzzJ4yn4MNg==", 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBrPgZmkf9WIxAA1eCaLkdytaQEIc0DT60e+w==", 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBuPgZmZnRLIRAAUMAP+Ux2za2z+Kuvx9EPPg==", 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.936 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBTPgZm8VqYBhAAmwz9Zhd1EMjy52y5RAJy3w==", 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBTPgZmzTfpDxAAVWrNCx4mKvts114/QRML4A==", 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBTPgZmLglsDBAAFvRmWjIVDH4e7vYpj54bNQ==", 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.938 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBTPgZm1zPTFRAAMh+bKVY1N263z+C/axveaQ==", 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBSPgZmtLWPCxAAUDwWCccb+H24IstFtiIyDw==", 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-mds", 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQB0PgZmRl59GhAAo34LKHHbIGg+xGejdAzLbA==", 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.940 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-mds" 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-mgr", 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQB0PgZm3WZ9GhAANcHz/HGCiGWEL7EildqWoQ==", 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-osd", 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQB0PgZmn219GhAArdEMUXDABABpDdgxaRVQlA==", 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-osd" 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rbd", 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQB0PgZmCXR9GhAAkiJe2YXae9518QO2wH4ang==", 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQB0PgZmqHp9GhAAPQ6zRDQl/1lcCRjejCM0cA==", 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rgw", 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQB0PgZmTIN9GhAAWbGY/uiHDAVo9QZXGoforQ==", 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBSPgZmEpVjJhAA73QsPidRWekJsp1yxqYzOQ==", 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.943 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:24.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:24.944 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-03-29T04:08:24.944 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQBSPgZmitfHIBAARJO706tkS1jlAHRtaoWd3Q==", 2024-03-29T04:08:24.944 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-03-29T04:08:24.944 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-03-29T04:08:24.944 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-03-29T04:08:24.944 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-03-29T04:08:24.944 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.944 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:24.944 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-03-29T04:08:24.944 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:08:24.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.951+0000 7f5b34582640 1 -- 172.21.15.97:0/2988256767 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5b1006c530 msgr2=0x7f5b1006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:24.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.952+0000 7f5b34582640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5b1006c530 0x7f5b1006e9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800b810 tx=0x7f5b28008040 comp rx=0 tx=0).stop 2024-03-29T04:08:24.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.952+0000 7f5b34582640 1 -- 172.21.15.97:0/2988256767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b2c15e9b0 msgr2=0x7f5b2c0c99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:24.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.952+0000 7f5b34582640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b2c15e9b0 0x7f5b2c0c99a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400c780 tx=0x7f5b2400cc40 comp rx=0 tx=0).stop 2024-03-29T04:08:24.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.952+0000 7f5b34582640 1 -- 172.21.15.97:0/2988256767 shutdown_connections 2024-03-29T04:08:24.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.952+0000 7f5b34582640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5b1006c530 0x7f5b1006e9f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.952+0000 7f5b34582640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b2c15e9b0 0x7f5b2c0c99a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.952+0000 7f5b34582640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b2c0d4f30 0x7f5b2c0c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.952+0000 7f5b34582640 1 --2- 172.21.15.97:0/2988256767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b2c0d2bf0 0x7f5b2c15d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:24.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.952+0000 7f5b34582640 1 -- 172.21.15.97:0/2988256767 >> 172.21.15.97:0/2988256767 conn(0x7f5b2c08bfc0 msgr2=0x7f5b2c0b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:24.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.952+0000 7f5b34582640 1 -- 172.21.15.97:0/2988256767 shutdown_connections 2024-03-29T04:08:24.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:24.952+0000 7f5b34582640 1 -- 172.21.15.97:0/2988256767 wait complete. 2024-03-29T04:08:24.947 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-29T04:08:25.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.068+0000 7fbd77949640 1 Processor -- start 2024-03-29T04:08:25.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.068+0000 7fbd77949640 1 -- start start 2024-03-29T04:08:25.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.069+0000 7fbd77949640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd70161e00 0x7fbd700b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.069+0000 7fbd77949640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbd700cd280 0x7fbd700718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.069+0000 7fbd77949640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd700b42a0 con 0x7fbd700999a0 2024-03-29T04:08:25.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.069+0000 7fbd77949640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fbd701621e0 con 0x7fbd700cd280 2024-03-29T04:08:25.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.069+0000 7fbd77949640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbd700b4580 con 0x7fbd70161e00 2024-03-29T04:08:25.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.069+0000 7fbd75ee6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbd700cd280 0x7fbd700718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.069+0000 7fbd75ee6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbd700cd280 0x7fbd700718b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50088/0 (socket says 172.21.15.97:50088) 2024-03-29T04:08:25.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.069+0000 7fbd74ee4640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbd700999a0 0x7fbd700ccb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59122/0 (socket says 172.21.15.97:59122) 2024-03-29T04:08:25.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.069+0000 7fbd75ee6640 1 -- 172.21.15.97:0/2260189037 learned_addr learned my addr 172.21.15.97:0/2260189037 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:25.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.069+0000 7fbd756e5640 1 --2- 172.21.15.97:0/2260189037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd70161e00 0x7fbd700b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.070+0000 7fbd75ee6640 1 -- 172.21.15.97:0/2260189037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd70161e00 msgr2=0x7fbd700b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:25.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.070+0000 7fbd75ee6640 1 --2- 172.21.15.97:0/2260189037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd70161e00 0x7fbd700b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.070+0000 7fbd75ee6640 1 -- 172.21.15.97:0/2260189037 >> v1:172.21.15.97:6789/0 conn(0x7fbd700999a0 legacy=0x7fbd700ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.070+0000 7fbd75ee6640 1 -- 172.21.15.97:0/2260189037 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd700b4860 con 0x7fbd700cd280 2024-03-29T04:08:25.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.070+0000 7fbd75ee6640 1 --2- 172.21.15.97:0/2260189037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd700cd280 0x7fbd700718b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009e80 tx=0x7fbd6802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16277c0a9c2ade server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:25.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.070+0000 7fbd67fff640 1 -- 172.21.15.97:0/2260189037 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6802caf0 con 0x7fbd700cd280 2024-03-29T04:08:25.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.070+0000 7fbd67fff640 1 -- 172.21.15.97:0/2260189037 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6802cc90 con 0x7fbd700cd280 2024-03-29T04:08:25.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.070+0000 7fbd67fff640 1 -- 172.21.15.97:0/2260189037 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68034610 con 0x7fbd700cd280 2024-03-29T04:08:25.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.071+0000 7fbd77949640 1 -- 172.21.15.97:0/2260189037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd700cd280 msgr2=0x7fbd700718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.071+0000 7fbd77949640 1 --2- 172.21.15.97:0/2260189037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd700cd280 0x7fbd700718b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009e80 tx=0x7fbd6802bf50 comp rx=0 tx=0).stop 2024-03-29T04:08:25.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.071+0000 7fbd77949640 1 -- 172.21.15.97:0/2260189037 shutdown_connections 2024-03-29T04:08:25.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.071+0000 7fbd77949640 1 --2- 172.21.15.97:0/2260189037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd700cd280 0x7fbd700718b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.071+0000 7fbd77949640 1 --2- 172.21.15.97:0/2260189037 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd70161e00 0x7fbd700b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.071+0000 7fbd77949640 1 -- 172.21.15.97:0/2260189037 >> 172.21.15.97:0/2260189037 conn(0x7fbd7006b510 msgr2=0x7fbd7006b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:25.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.071+0000 7fbd77949640 1 -- 172.21.15.97:0/2260189037 shutdown_connections 2024-03-29T04:08:25.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.071+0000 7fbd77949640 1 -- 172.21.15.97:0/2260189037 wait complete. 2024-03-29T04:08:25.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.072+0000 7fbd77949640 1 Processor -- start 2024-03-29T04:08:25.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.072+0000 7fbd77949640 1 -- start start 2024-03-29T04:08:25.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd77949640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd700999a0 0x7fbd7014f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd77949640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd700cd280 0x7fbd700cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd77949640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd70161e00 0x7fbd7006e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd77949640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70071df0 con 0x7fbd700999a0 2024-03-29T04:08:25.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd77949640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70099e00 con 0x7fbd700cd280 2024-03-29T04:08:25.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd77949640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fbd70071f70 con 0x7fbd70161e00 2024-03-29T04:08:25.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd756e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd700999a0 0x7fbd7014f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd756e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd700999a0 0x7fbd7014f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50100/0 (socket says 172.21.15.97:50100) 2024-03-29T04:08:25.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd756e5640 1 -- 172.21.15.97:0/2555477726 learned_addr learned my addr 172.21.15.97:0/2555477726 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:25.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd75ee6640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd70161e00 0x7fbd7006e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd74ee4640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd700cd280 0x7fbd700cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd756e5640 1 -- 172.21.15.97:0/2555477726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd70161e00 msgr2=0x7fbd7006e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd756e5640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd70161e00 0x7fbd7006e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd756e5640 1 -- 172.21.15.97:0/2555477726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd700cd280 msgr2=0x7fbd700cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd756e5640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd700cd280 0x7fbd700cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd756e5640 1 -- 172.21.15.97:0/2555477726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7014fc40 con 0x7fbd700999a0 2024-03-29T04:08:25.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.073+0000 7fbd756e5640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd700999a0 0x7fbd7014f700 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00d620 tx=0x7fbd6c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:25.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.074+0000 7fbd65ffb640 1 -- 172.21.15.97:0/2555477726 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00b620 con 0x7fbd700999a0 2024-03-29T04:08:25.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.074+0000 7fbd65ffb640 1 -- 172.21.15.97:0/2555477726 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6c019070 con 0x7fbd700999a0 2024-03-29T04:08:25.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.074+0000 7fbd77949640 1 -- 172.21.15.97:0/2555477726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd700cf9e0 con 0x7fbd700999a0 2024-03-29T04:08:25.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.074+0000 7fbd65ffb640 1 -- 172.21.15.97:0/2555477726 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00ba00 con 0x7fbd700999a0 2024-03-29T04:08:25.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.074+0000 7fbd77949640 1 -- 172.21.15.97:0/2555477726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd700ad230 con 0x7fbd700999a0 2024-03-29T04:08:25.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.075+0000 7fbd77949640 1 -- 172.21.15.97:0/2555477726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd70068640 con 0x7fbd700999a0 2024-03-29T04:08:25.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.075+0000 7fbd65ffb640 1 -- 172.21.15.97:0/2555477726 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbd6c00bba0 con 0x7fbd700999a0 2024-03-29T04:08:25.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.075+0000 7fbd65ffb640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fbd5406c560 0x7fbd5406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.076+0000 7fbd65ffb640 1 -- 172.21.15.97:0/2555477726 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fbd6c08b740 con 0x7fbd700999a0 2024-03-29T04:08:25.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.076+0000 7fbd74ee4640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fbd5406c560 0x7fbd5406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.076+0000 7fbd74ee4640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fbd5406c560 0x7fbd5406ea20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbd70160a20 tx=0x7fbd60007cc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:25.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.080+0000 7fbd65ffb640 1 -- 172.21.15.97:0/2555477726 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbd7014fc40 con 0x7fbd700999a0 2024-03-29T04:08:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.319+0000 7fbd77949640 1 -- 172.21.15.97:0/2555477726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fbd700b1b20 con 0x7fbd700999a0 2024-03-29T04:08:25.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.321+0000 7fbd65ffb640 1 -- 172.21.15.97:0/2555477726 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7fbd6c05a0b0 con 0x7fbd700999a0 2024-03-29T04:08:25.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.324+0000 7fbd77949640 1 -- 172.21.15.97:0/2555477726 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fbd5406c560 msgr2=0x7fbd5406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.324+0000 7fbd77949640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fbd5406c560 0x7fbd5406ea20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbd70160a20 tx=0x7fbd60007cc0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.324+0000 7fbd77949640 1 -- 172.21.15.97:0/2555477726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd700999a0 msgr2=0x7fbd7014f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.324+0000 7fbd77949640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd700999a0 0x7fbd7014f700 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00d620 tx=0x7fbd6c00dae0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.324+0000 7fbd77949640 1 -- 172.21.15.97:0/2555477726 shutdown_connections 2024-03-29T04:08:25.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.324+0000 7fbd77949640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fbd5406c560 0x7fbd5406ea20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.324+0000 7fbd77949640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbd70161e00 0x7fbd7006e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.324+0000 7fbd77949640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd700cd280 0x7fbd700cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.324+0000 7fbd77949640 1 --2- 172.21.15.97:0/2555477726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd700999a0 0x7fbd7014f700 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.324+0000 7fbd77949640 1 -- 172.21.15.97:0/2555477726 >> 172.21.15.97:0/2555477726 conn(0x7fbd7006b510 msgr2=0x7fbd700cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:25.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.324+0000 7fbd77949640 1 -- 172.21.15.97:0/2555477726 shutdown_connections 2024-03-29T04:08:25.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.324+0000 7fbd77949640 1 -- 172.21.15.97:0/2555477726 wait complete. 2024-03-29T04:08:25.314 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-29T04:08:25.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.437+0000 7f77b2e3c640 1 Processor -- start 2024-03-29T04:08:25.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.438+0000 7f77b2e3c640 1 -- start start 2024-03-29T04:08:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.438+0000 7f77b2e3c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f77ac15e780 0x7f77ac0cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.438+0000 7f77b2e3c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f77ac15eb60 0x7f77ac0cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.438+0000 7f77b2e3c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77ac0cfe70 con 0x7f77ac0cf640 2024-03-29T04:08:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.438+0000 7f77b2e3c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f77ac0b7f50 con 0x7f77ac15e780 2024-03-29T04:08:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.438+0000 7f77b2e3c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f77ac0b8230 con 0x7f77ac15eb60 2024-03-29T04:08:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.438+0000 7f77b0bd8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f77ac15e780 0x7f77ac0cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.438+0000 7f77b13d9640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f77ac0cf640 0x7f77ac0b8ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59134/0 (socket says 172.21.15.97:59134) 2024-03-29T04:08:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.438+0000 7f77b0bd8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f77ac15e780 0x7f77ac0cebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50104/0 (socket says 172.21.15.97:50104) 2024-03-29T04:08:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.438+0000 7f77b13d9640 1 -- 172.21.15.97:0/1686279430 learned_addr learned my addr 172.21.15.97:0/1686279430 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:25.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.439+0000 7f77abfff640 1 --2- 172.21.15.97:0/1686279430 >> v2:172.21.15.154:3300/0 conn(0x7f77ac15eb60 0x7f77ac0cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.439+0000 7f77ab7fe640 1 -- 172.21.15.97:0/1686279430 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2933858394 0 0) 0x7f77ac0cfe70 con 0x7f77ac0cf640 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.439+0000 7f77ab7fe640 1 -- 172.21.15.97:0/1686279430 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7794003520 con 0x7f77ac0cf640 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.439+0000 7f77b0bd8640 1 -- 172.21.15.97:0/1686279430 >> v2:172.21.15.154:3300/0 conn(0x7f77ac15eb60 msgr2=0x7f77ac0cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.439+0000 7f77b0bd8640 1 --2- 172.21.15.97:0/1686279430 >> v2:172.21.15.154:3300/0 conn(0x7f77ac15eb60 0x7f77ac0cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.439+0000 7f77b0bd8640 1 -- 172.21.15.97:0/1686279430 >> v1:172.21.15.97:6789/0 conn(0x7f77ac0cf640 legacy=0x7f77ac0b8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.439+0000 7f77b0bd8640 1 -- 172.21.15.97:0/1686279430 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac0b8510 con 0x7f77ac15e780 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.439+0000 7f77b0bd8640 1 --2- 172.21.15.97:0/1686279430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ac15e780 0x7f77ac0cebc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0d2020 tx=0x7f779c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9c4bdb0c025e9f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.440+0000 7f77ab7fe640 1 -- 172.21.15.97:0/1686279430 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c035020 con 0x7f77ac15e780 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.440+0000 7f77ab7fe640 1 -- 172.21.15.97:0/1686279430 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f779c031830 con 0x7f77ac15e780 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.440+0000 7f77ab7fe640 1 -- 172.21.15.97:0/1686279430 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c031c00 con 0x7f77ac15e780 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.440+0000 7f77b2e3c640 1 -- 172.21.15.97:0/1686279430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ac15e780 msgr2=0x7f77ac0cebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.440+0000 7f77b2e3c640 1 --2- 172.21.15.97:0/1686279430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ac15e780 0x7f77ac0cebc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0d2020 tx=0x7f779c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.440+0000 7f77b2e3c640 1 -- 172.21.15.97:0/1686279430 shutdown_connections 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.440+0000 7f77b2e3c640 1 --2- 172.21.15.97:0/1686279430 >> v2:172.21.15.154:3300/0 conn(0x7f77ac15eb60 0x7f77ac0cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.440+0000 7f77b2e3c640 1 --2- 172.21.15.97:0/1686279430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ac15e780 0x7f77ac0cebc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.440+0000 7f77b2e3c640 1 -- 172.21.15.97:0/1686279430 >> 172.21.15.97:0/1686279430 conn(0x7f77ac08bfc0 msgr2=0x7f77ac0d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:25.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.440+0000 7f77b2e3c640 1 -- 172.21.15.97:0/1686279430 shutdown_connections 2024-03-29T04:08:25.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.440+0000 7f77b2e3c640 1 -- 172.21.15.97:0/1686279430 wait complete. 2024-03-29T04:08:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.441+0000 7f77b2e3c640 1 Processor -- start 2024-03-29T04:08:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.441+0000 7f77b2e3c640 1 -- start start 2024-03-29T04:08:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.442+0000 7f77b2e3c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77ac0cf640 0x7f77ac151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.442+0000 7f77b2e3c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77ac15e780 0x7f77ac1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.442+0000 7f77b2e3c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ac0d4640 0x7f77ac0a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.442+0000 7f77b2e3c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac0d3580 con 0x7f77ac0d4640 2024-03-29T04:08:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.442+0000 7f77b2e3c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac0b8ad0 con 0x7f77ac15e780 2024-03-29T04:08:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.442+0000 7f77b2e3c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f77ac0d3700 con 0x7f77ac0cf640 2024-03-29T04:08:25.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.442+0000 7f77b13d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ac0d4640 0x7f77ac0a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.442+0000 7f77b13d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ac0d4640 0x7f77ac0a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50112/0 (socket says 172.21.15.97:50112) 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.442+0000 7f77b13d9640 1 -- 172.21.15.97:0/3611911045 learned_addr learned my addr 172.21.15.97:0/3611911045 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.442+0000 7f77abfff640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77ac15e780 0x7f77ac1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.443+0000 7f77b0bd8640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77ac0cf640 0x7f77ac151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.443+0000 7f77b13d9640 1 -- 172.21.15.97:0/3611911045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77ac0cf640 msgr2=0x7f77ac151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.443+0000 7f77b13d9640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77ac0cf640 0x7f77ac151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.443+0000 7f77b13d9640 1 -- 172.21.15.97:0/3611911045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77ac15e780 msgr2=0x7f77ac1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.443+0000 7f77b13d9640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77ac15e780 0x7f77ac1609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.443+0000 7f77b13d9640 1 -- 172.21.15.97:0/3611911045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac1618c0 con 0x7f77ac0d4640 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.443+0000 7f77b13d9640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ac0d4640 0x7f77ac0a0fc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f77a400cb70 tx=0x7f77a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.443+0000 7f77a97fa640 1 -- 172.21.15.97:0/3611911045 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4017070 con 0x7f77ac0d4640 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.443+0000 7f77a97fa640 1 -- 172.21.15.97:0/3611911045 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a400ace0 con 0x7f77ac0d4640 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.443+0000 7f77b2e3c640 1 -- 172.21.15.97:0/3611911045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ac161300 con 0x7f77ac0d4640 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.443+0000 7f77a97fa640 1 -- 172.21.15.97:0/3611911045 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4012620 con 0x7f77ac0d4640 2024-03-29T04:08:25.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.443+0000 7f77b2e3c640 1 -- 172.21.15.97:0/3611911045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ac09e7b0 con 0x7f77ac0d4640 2024-03-29T04:08:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.444+0000 7f77b2e3c640 1 -- 172.21.15.97:0/3611911045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77ac068640 con 0x7f77ac0d4640 2024-03-29T04:08:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.445+0000 7f77a97fa640 1 -- 172.21.15.97:0/3611911045 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f77a401a070 con 0x7f77ac0d4640 2024-03-29T04:08:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.445+0000 7f77a97fa640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f779806c530 0x7f779806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.445+0000 7f77a97fa640 1 -- 172.21.15.97:0/3611911045 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f77a408b080 con 0x7f77ac0d4640 2024-03-29T04:08:25.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.445+0000 7f77b0bd8640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f779806c530 0x7f779806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.445+0000 7f77b0bd8640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f779806c530 0x7f779806e9f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f779c036000 tx=0x7f779c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:25.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.449+0000 7f77a97fa640 1 -- 172.21.15.97:0/3611911045 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f77a4059430 con 0x7f77ac0d4640 2024-03-29T04:08:25.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.698+0000 7f77b2e3c640 1 -- 172.21.15.97:0/3611911045 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f77ac06b510 con 0x7f77ac0d4640 2024-03-29T04:08:25.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.700+0000 7f77a97fa640 1 -- 172.21.15.97:0/3611911045 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f77a405d730 con 0x7f77ac0d4640 2024-03-29T04:08:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.701+0000 7f77b2e3c640 1 -- 172.21.15.97:0/3611911045 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f779806c530 msgr2=0x7f779806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.701+0000 7f77b2e3c640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f779806c530 0x7f779806e9f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f779c036000 tx=0x7f779c008030 comp rx=0 tx=0).stop 2024-03-29T04:08:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.701+0000 7f77b2e3c640 1 -- 172.21.15.97:0/3611911045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ac0d4640 msgr2=0x7f77ac0a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.701+0000 7f77b2e3c640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ac0d4640 0x7f77ac0a0fc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f77a400cb70 tx=0x7f77a400a430 comp rx=0 tx=0).stop 2024-03-29T04:08:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.702+0000 7f77b2e3c640 1 -- 172.21.15.97:0/3611911045 shutdown_connections 2024-03-29T04:08:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.702+0000 7f77b2e3c640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f779806c530 0x7f779806e9f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.702+0000 7f77b2e3c640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ac0d4640 0x7f77ac0a0fc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.702+0000 7f77b2e3c640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77ac15e780 0x7f77ac1609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.702+0000 7f77b2e3c640 1 --2- 172.21.15.97:0/3611911045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77ac0cf640 0x7f77ac151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.702+0000 7f77b2e3c640 1 -- 172.21.15.97:0/3611911045 >> 172.21.15.97:0/3611911045 conn(0x7f77ac08bfc0 msgr2=0x7f77ac06d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.702+0000 7f77b2e3c640 1 -- 172.21.15.97:0/3611911045 shutdown_connections 2024-03-29T04:08:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.702+0000 7f77b2e3c640 1 -- 172.21.15.97:0/3611911045 wait complete. 2024-03-29T04:08:25.692 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-29T04:08:25.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.814+0000 7faa9de88640 1 Processor -- start 2024-03-29T04:08:25.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.814+0000 7faa9de88640 1 -- start start 2024-03-29T04:08:25.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.814+0000 7faa9de88640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faa9815eb60 0x7faa980cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.814+0000 7faa9de88640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa980ba000 0x7faa980d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.814+0000 7faa9de88640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa980cfe70 con 0x7faa9815e780 2024-03-29T04:08:25.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.814+0000 7faa9de88640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7faa980b7f50 con 0x7faa9815eb60 2024-03-29T04:08:25.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.814+0000 7faa9de88640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7faa980b8230 con 0x7faa980ba000 2024-03-29T04:08:25.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa977fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7faa9815e780 0x7faa980cebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59138/0 (socket says 172.21.15.97:59138) 2024-03-29T04:08:25.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa977fe640 1 -- 172.21.15.97:0/1189573671 learned_addr learned my addr 172.21.15.97:0/1189573671 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:25.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa97fff640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa980ba000 0x7faa980d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa96ffd640 1 --2- 172.21.15.97:0/1189573671 >> v2:172.21.15.154:3300/0 conn(0x7faa9815eb60 0x7faa980cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 708544648 0 0) 0x7faa980cfe70 con 0x7faa9815e780 2024-03-29T04:08:25.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa80003520 con 0x7faa9815e780 2024-03-29T04:08:25.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1832369897 0 0) 0x7faa80003520 con 0x7faa9815e780 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa980ba000 msgr2=0x7faa980d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa980ba000 0x7faa980d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 >> v2:172.21.15.154:3300/0 conn(0x7faa9815eb60 msgr2=0x7faa980cf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 --2- 172.21.15.97:0/1189573671 >> v2:172.21.15.154:3300/0 conn(0x7faa9815eb60 0x7faa980cf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa980b8510 con 0x7faa9815e780 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7faa88001670 con 0x7faa9815e780 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 >> v1:172.21.15.97:6789/0 conn(0x7faa9815e780 legacy=0x7faa980cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa800039b0 0x7faa80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa80004800 0x7faa80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa800051c0 0x7faa8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faa980b7f50 con 0x7faa800039b0 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faa980b8230 con 0x7faa800051c0 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7faa8000bf00 con 0x7faa80004800 2024-03-29T04:08:25.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa97fff640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa980ba000 0x7faa980d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa977fe640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa800039b0 0x7faa80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.815+0000 7faa96ffd640 1 --2- 172.21.15.97:0/1189573671 >> v2:172.21.15.154:3300/0 conn(0x7faa9815eb60 0x7faa980cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.816+0000 7faa96ffd640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa80004800 0x7faa80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.816+0000 7faa97fff640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa800051c0 0x7faa8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.816+0000 7faa977fe640 1 -- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa80004800 msgr2=0x7faa80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.816+0000 7faa977fe640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa80004800 0x7faa80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.816+0000 7faa977fe640 1 -- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa800051c0 msgr2=0x7faa8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.816+0000 7faa977fe640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa800051c0 0x7faa8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.816+0000 7faa977fe640 1 -- 172.21.15.97:0/1189573671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa80003520 con 0x7faa800039b0 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.816+0000 7faa97fff640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa800051c0 0x7faa8000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.816+0000 7faa96ffd640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa80004800 0x7faa80004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.816+0000 7faa977fe640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa800039b0 0x7faa80003dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7faa8800c270 tx=0x7faa80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef1b6264217e037 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.816+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88007d40 con 0x7faa800039b0 2024-03-29T04:08:25.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.816+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa88030070 con 0x7faa800039b0 2024-03-29T04:08:25.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.816+0000 7faa967fc640 1 -- 172.21.15.97:0/1189573671 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8802cd40 con 0x7faa800039b0 2024-03-29T04:08:25.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.817+0000 7faa9de88640 1 -- 172.21.15.97:0/1189573671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa800039b0 msgr2=0x7faa80003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.817+0000 7faa9de88640 1 --2- 172.21.15.97:0/1189573671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa800039b0 0x7faa80003dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7faa8800c270 tx=0x7faa80005830 comp rx=0 tx=0).stop 2024-03-29T04:08:25.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.817+0000 7faa97fff640 1 -- 172.21.15.97:0/1189573671 reap_dead start 2024-03-29T04:08:25.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.817+0000 7faa9de88640 1 -- 172.21.15.97:0/1189573671 shutdown_connections 2024-03-29T04:08:25.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.817+0000 7faa9de88640 1 -- 172.21.15.97:0/1189573671 >> 172.21.15.97:0/1189573671 conn(0x7faa9808bfc0 msgr2=0x7faa980d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:25.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.817+0000 7faa9de88640 1 -- 172.21.15.97:0/1189573671 shutdown_connections 2024-03-29T04:08:25.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.817+0000 7faa9de88640 1 -- 172.21.15.97:0/1189573671 wait complete. 2024-03-29T04:08:25.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.818+0000 7faa9de88640 1 Processor -- start 2024-03-29T04:08:25.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.818+0000 7faa9de88640 1 -- start start 2024-03-29T04:08:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa9de88640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa800039b0 0x7faa9815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa9de88640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa981f7940 0x7faa9814d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa9de88640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa981f7dc0 0x7faa9814de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa9de88640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faa980cf810 con 0x7faa981f7940 2024-03-29T04:08:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa9de88640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faa980b8ad0 con 0x7faa981f7dc0 2024-03-29T04:08:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa9de88640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7faa980cf990 con 0x7faa800039b0 2024-03-29T04:08:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa96ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa981f7940 0x7faa9814d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa96ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa981f7940 0x7faa9814d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50126/0 (socket says 172.21.15.97:50126) 2024-03-29T04:08:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa96ffd640 1 -- 172.21.15.97:0/943760674 learned_addr learned my addr 172.21.15.97:0/943760674 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:25.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa977fe640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa800039b0 0x7faa9815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa97fff640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa981f7dc0 0x7faa9814de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa96ffd640 1 -- 172.21.15.97:0/943760674 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa800039b0 msgr2=0x7faa9815b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa96ffd640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa800039b0 0x7faa9815b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa96ffd640 1 -- 172.21.15.97:0/943760674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa981f7dc0 msgr2=0x7faa9814de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa96ffd640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa981f7dc0 0x7faa9814de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.819+0000 7faa96ffd640 1 -- 172.21.15.97:0/943760674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa981f8390 con 0x7faa981f7940 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.820+0000 7faa96ffd640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa981f7940 0x7faa9814d950 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0109a0 tx=0x7faa8c010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.820+0000 7faa7ffff640 1 -- 172.21.15.97:0/943760674 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c00f040 con 0x7faa981f7940 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.820+0000 7faa7ffff640 1 -- 172.21.15.97:0/943760674 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c00eba0 con 0x7faa981f7940 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.820+0000 7faa9de88640 1 -- 172.21.15.97:0/943760674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa981f8670 con 0x7faa981f7940 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.820+0000 7faa7ffff640 1 -- 172.21.15.97:0/943760674 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c01c400 con 0x7faa981f7940 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.820+0000 7faa9de88640 1 -- 172.21.15.97:0/943760674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9815a780 con 0x7faa981f7940 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.821+0000 7faa9de88640 1 -- 172.21.15.97:0/943760674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa98068640 con 0x7faa981f7940 2024-03-29T04:08:25.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.821+0000 7faa7ffff640 1 -- 172.21.15.97:0/943760674 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faa8c01c5a0 con 0x7faa981f7940 2024-03-29T04:08:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.822+0000 7faa7ffff640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7faa8406c560 0x7faa8406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.822+0000 7faa7ffff640 1 -- 172.21.15.97:0/943760674 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7faa8c08b870 con 0x7faa981f7940 2024-03-29T04:08:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.822+0000 7faa977fe640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7faa8406c560 0x7faa8406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.822+0000 7faa977fe640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7faa8406c560 0x7faa8406ea20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7faa88034000 tx=0x7faa88007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:25.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:25.826+0000 7faa7ffff640 1 -- 172.21.15.97:0/943760674 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faa8c018030 con 0x7faa981f7940 2024-03-29T04:08:26.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.068+0000 7faa9de88640 1 -- 172.21.15.97:0/943760674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7faa981f8840 con 0x7faa981f7940 2024-03-29T04:08:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.072+0000 7faa7ffff640 1 -- 172.21.15.97:0/943760674 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7faa8c059b90 con 0x7faa981f7940 2024-03-29T04:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.074+0000 7faa9de88640 1 -- 172.21.15.97:0/943760674 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7faa8406c560 msgr2=0x7faa8406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.074+0000 7faa9de88640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7faa8406c560 0x7faa8406ea20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7faa88034000 tx=0x7faa88007000 comp rx=0 tx=0).stop 2024-03-29T04:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.074+0000 7faa9de88640 1 -- 172.21.15.97:0/943760674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa981f7940 msgr2=0x7faa9814d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.074+0000 7faa9de88640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa981f7940 0x7faa9814d950 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0109a0 tx=0x7faa8c010e60 comp rx=0 tx=0).stop 2024-03-29T04:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.074+0000 7faa9de88640 1 -- 172.21.15.97:0/943760674 shutdown_connections 2024-03-29T04:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.074+0000 7faa9de88640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7faa8406c560 0x7faa8406ea20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.074+0000 7faa9de88640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa981f7dc0 0x7faa9814de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.074+0000 7faa9de88640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa981f7940 0x7faa9814d950 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.074+0000 7faa9de88640 1 --2- 172.21.15.97:0/943760674 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa800039b0 0x7faa9815b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.074+0000 7faa9de88640 1 -- 172.21.15.97:0/943760674 >> 172.21.15.97:0/943760674 conn(0x7faa9808bfc0 msgr2=0x7faa9806ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.074+0000 7faa9de88640 1 -- 172.21.15.97:0/943760674 shutdown_connections 2024-03-29T04:08:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.074+0000 7faa9de88640 1 -- 172.21.15.97:0/943760674 wait complete. 2024-03-29T04:08:26.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-29T04:08:26.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.186+0000 7f367a59d640 1 Processor -- start 2024-03-29T04:08:26.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.186+0000 7f367a59d640 1 -- start start 2024-03-29T04:08:26.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.187+0000 7f367a59d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f367415ec10 0x7f36740c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.187+0000 7f367a59d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f367415f100 con 0x7f36740d0730 2024-03-29T04:08:26.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.187+0000 7f367a59d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36740cebd0 con 0x7f36740ce740 2024-03-29T04:08:26.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.187+0000 7f367a59d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f367415f370 con 0x7f367415ec10 2024-03-29T04:08:26.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.187+0000 7f3678b3a640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f36740d0730 0x7f36740b30b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59142/0 (socket says 172.21.15.97:59142) 2024-03-29T04:08:26.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.187+0000 7f3678b3a640 1 -- 172.21.15.97:0/420915857 learned_addr learned my addr 172.21.15.97:0/420915857 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:26.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.187+0000 7f3673fff640 1 --2- 172.21.15.97:0/420915857 >> v2:172.21.15.154:3300/0 conn(0x7f367415ec10 0x7f36740c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.187+0000 7f3672ffd640 1 -- 172.21.15.97:0/420915857 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3957169378 0 0) 0x7f367415f100 con 0x7f36740d0730 2024-03-29T04:08:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.187+0000 7f3672ffd640 1 -- 172.21.15.97:0/420915857 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f365c003520 con 0x7f36740d0730 2024-03-29T04:08:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.187+0000 7f3673fff640 1 -- 172.21.15.97:0/420915857 >> v1:172.21.15.154:6789/0 conn(0x7f36740ce740 legacy=0x7f36740d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.187+0000 7f3673fff640 1 -- 172.21.15.97:0/420915857 >> v1:172.21.15.97:6789/0 conn(0x7f36740d0730 legacy=0x7f36740b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.187+0000 7f3673fff640 1 -- 172.21.15.97:0/420915857 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367415f650 con 0x7f367415ec10 2024-03-29T04:08:26.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.188+0000 7f3673fff640 1 --2- 172.21.15.97:0/420915857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f367415ec10 0x7f36740c67e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f36640097a0 tx=0x7f366402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a32b7dde6c00c49 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:26.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.188+0000 7f3672ffd640 1 -- 172.21.15.97:0/420915857 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664035020 con 0x7f367415ec10 2024-03-29T04:08:26.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.188+0000 7f3672ffd640 1 -- 172.21.15.97:0/420915857 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3664031830 con 0x7f367415ec10 2024-03-29T04:08:26.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.188+0000 7f3672ffd640 1 -- 172.21.15.97:0/420915857 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664031be0 con 0x7f367415ec10 2024-03-29T04:08:26.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.189+0000 7f367a59d640 1 -- 172.21.15.97:0/420915857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f367415ec10 msgr2=0x7f36740c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.189+0000 7f367a59d640 1 --2- 172.21.15.97:0/420915857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f367415ec10 0x7f36740c67e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f36640097a0 tx=0x7f366402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.189+0000 7f367a59d640 1 -- 172.21.15.97:0/420915857 shutdown_connections 2024-03-29T04:08:26.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.189+0000 7f367a59d640 1 --2- 172.21.15.97:0/420915857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f367415ec10 0x7f36740c67e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.189+0000 7f367a59d640 1 -- 172.21.15.97:0/420915857 >> 172.21.15.97:0/420915857 conn(0x7f367406b510 msgr2=0x7f367406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:26.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.189+0000 7f367a59d640 1 -- 172.21.15.97:0/420915857 shutdown_connections 2024-03-29T04:08:26.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.189+0000 7f367a59d640 1 -- 172.21.15.97:0/420915857 wait complete. 2024-03-29T04:08:26.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.190+0000 7f367a59d640 1 Processor -- start 2024-03-29T04:08:26.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.190+0000 7f367a59d640 1 -- start start 2024-03-29T04:08:26.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f367a59d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36740ce740 0x7f367415a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f367a59d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36740d0730 0x7f367406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f367a59d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f367415ec10 0x7f3674072700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f367a59d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f367415fe50 con 0x7f36740ce740 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f367a59d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f36740c6ee0 con 0x7f367415ec10 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f367a59d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f367415ffd0 con 0x7f36740d0730 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f3673fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36740ce740 0x7f367415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f3673fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36740ce740 0x7f367415a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50142/0 (socket says 172.21.15.97:50142) 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f3673fff640 1 -- 172.21.15.97:0/1981299229 learned_addr learned my addr 172.21.15.97:0/1981299229 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f3678b3a640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f367415ec10 0x7f3674072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f36737fe640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36740d0730 0x7f367406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f3673fff640 1 -- 172.21.15.97:0/1981299229 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36740d0730 msgr2=0x7f367406d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f3673fff640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36740d0730 0x7f367406d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f3673fff640 1 -- 172.21.15.97:0/1981299229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f367415ec10 msgr2=0x7f3674072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f3673fff640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f367415ec10 0x7f3674072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f3673fff640 1 -- 172.21.15.97:0/1981299229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367406e1d0 con 0x7f36740ce740 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.191+0000 7f3673fff640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36740ce740 0x7f367415a360 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3664034040 tx=0x7f36640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.192+0000 7f3670ff9640 1 -- 172.21.15.97:0/1981299229 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664004230 con 0x7f36740ce740 2024-03-29T04:08:26.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.192+0000 7f3670ff9640 1 -- 172.21.15.97:0/1981299229 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36640043d0 con 0x7f36740ce740 2024-03-29T04:08:26.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.192+0000 7f367a59d640 1 -- 172.21.15.97:0/1981299229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367406e4b0 con 0x7f36740ce740 2024-03-29T04:08:26.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.192+0000 7f3670ff9640 1 -- 172.21.15.97:0/1981299229 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366403a870 con 0x7f36740ce740 2024-03-29T04:08:26.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.192+0000 7f367a59d640 1 -- 172.21.15.97:0/1981299229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36741f56a0 con 0x7f36740ce740 2024-03-29T04:08:26.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.193+0000 7f367a59d640 1 -- 172.21.15.97:0/1981299229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3674068640 con 0x7f36740ce740 2024-03-29T04:08:26.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.193+0000 7f3670ff9640 1 -- 172.21.15.97:0/1981299229 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3664038020 con 0x7f36740ce740 2024-03-29T04:08:26.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.193+0000 7f3670ff9640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f366006c560 0x7f366006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.193+0000 7f3670ff9640 1 -- 172.21.15.97:0/1981299229 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f36640ace90 con 0x7f36740ce740 2024-03-29T04:08:26.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.193+0000 7f36737fe640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f366006c560 0x7f366006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.194+0000 7f36737fe640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f366006c560 0x7f366006ea20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f367415a8a0 tx=0x7f3668009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:26.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.198+0000 7f3670ff9640 1 -- 172.21.15.97:0/1981299229 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3664040070 con 0x7f36740ce740 2024-03-29T04:08:26.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.441+0000 7f367a59d640 1 -- 172.21.15.97:0/1981299229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f36740b4f60 con 0x7f36740ce740 2024-03-29T04:08:26.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.443+0000 7f3670ff9640 1 -- 172.21.15.97:0/1981299229 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f366407b1b0 con 0x7f36740ce740 2024-03-29T04:08:26.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.445+0000 7f367a59d640 1 -- 172.21.15.97:0/1981299229 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f366006c560 msgr2=0x7f366006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.445+0000 7f367a59d640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f366006c560 0x7f366006ea20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f367415a8a0 tx=0x7f3668009040 comp rx=0 tx=0).stop 2024-03-29T04:08:26.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.445+0000 7f367a59d640 1 -- 172.21.15.97:0/1981299229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36740ce740 msgr2=0x7f367415a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.445+0000 7f367a59d640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36740ce740 0x7f367415a360 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3664034040 tx=0x7f36640046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.445+0000 7f367a59d640 1 -- 172.21.15.97:0/1981299229 shutdown_connections 2024-03-29T04:08:26.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.445+0000 7f367a59d640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f366006c560 0x7f366006ea20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.445+0000 7f367a59d640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f367415ec10 0x7f3674072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.445+0000 7f367a59d640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36740d0730 0x7f367406d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.445+0000 7f367a59d640 1 --2- 172.21.15.97:0/1981299229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36740ce740 0x7f367415a360 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.445+0000 7f367a59d640 1 -- 172.21.15.97:0/1981299229 >> 172.21.15.97:0/1981299229 conn(0x7f367406b510 msgr2=0x7f36740ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:26.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.445+0000 7f367a59d640 1 -- 172.21.15.97:0/1981299229 shutdown_connections 2024-03-29T04:08:26.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.445+0000 7f367a59d640 1 -- 172.21.15.97:0/1981299229 wait complete. 2024-03-29T04:08:26.436 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-29T04:08:26.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.558+0000 7f6c4584a640 1 Processor -- start 2024-03-29T04:08:26.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.558+0000 7f6c4584a640 1 -- start start 2024-03-29T04:08:26.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.559+0000 7f6c4584a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c4015c7e0 0x7f6c4009b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.559+0000 7f6c4584a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c4009bc60 0x7f6c4006faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.559+0000 7f6c4584a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c400d05a0 con 0x7f6c400d0030 2024-03-29T04:08:26.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.559+0000 7f6c4584a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f6c4009ac40 con 0x7f6c4015c7e0 2024-03-29T04:08:26.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.559+0000 7f6c4584a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6c4009af20 con 0x7f6c4009bc60 2024-03-29T04:08:26.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.559+0000 7f6c3effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c4015c7e0 0x7f6c4009b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.560+0000 7f6c3effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c4015c7e0 0x7f6c4009b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50144/0 (socket says 172.21.15.97:50144) 2024-03-29T04:08:26.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.560+0000 7f6c3effd640 1 -- 172.21.15.97:0/388262628 learned_addr learned my addr 172.21.15.97:0/388262628 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:26.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.560+0000 7f6c3e7fc640 1 --2- 172.21.15.97:0/388262628 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c4009bc60 0x7f6c4006faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.560+0000 7f6c3effd640 1 -- 172.21.15.97:0/388262628 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c4009bc60 msgr2=0x7f6c4006faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:26.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.560+0000 7f6c3effd640 1 --2- 172.21.15.97:0/388262628 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c4009bc60 0x7f6c4006faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.560+0000 7f6c3effd640 1 -- 172.21.15.97:0/388262628 >> v1:172.21.15.154:6789/0 conn(0x7f6c400d0030 legacy=0x7f6c401601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.560+0000 7f6c3effd640 1 -- 172.21.15.97:0/388262628 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4009b200 con 0x7f6c4015c7e0 2024-03-29T04:08:26.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.561+0000 7f6c3effd640 1 --2- 172.21.15.97:0/388262628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4015c7e0 0x7f6c4009b720 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009e50 tx=0x7f6c3002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caacc1affd106d42 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:26.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.561+0000 7f6c3dffb640 1 -- 172.21.15.97:0/388262628 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3002cb00 con 0x7f6c4015c7e0 2024-03-29T04:08:26.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.561+0000 7f6c3dffb640 1 -- 172.21.15.97:0/388262628 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c3002cca0 con 0x7f6c4015c7e0 2024-03-29T04:08:26.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.561+0000 7f6c3dffb640 1 -- 172.21.15.97:0/388262628 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30034610 con 0x7f6c4015c7e0 2024-03-29T04:08:26.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.561+0000 7f6c4584a640 1 -- 172.21.15.97:0/388262628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4015c7e0 msgr2=0x7f6c4009b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.561+0000 7f6c4584a640 1 --2- 172.21.15.97:0/388262628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4015c7e0 0x7f6c4009b720 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009e50 tx=0x7f6c3002bf60 comp rx=0 tx=0).stop 2024-03-29T04:08:26.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.561+0000 7f6c4584a640 1 -- 172.21.15.97:0/388262628 shutdown_connections 2024-03-29T04:08:26.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.561+0000 7f6c4584a640 1 --2- 172.21.15.97:0/388262628 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c4009bc60 0x7f6c4006faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.561+0000 7f6c4584a640 1 --2- 172.21.15.97:0/388262628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4015c7e0 0x7f6c4009b720 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.561+0000 7f6c4584a640 1 -- 172.21.15.97:0/388262628 >> 172.21.15.97:0/388262628 conn(0x7f6c400d2460 msgr2=0x7f6c400d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:26.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.561+0000 7f6c4584a640 1 -- 172.21.15.97:0/388262628 shutdown_connections 2024-03-29T04:08:26.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.561+0000 7f6c4584a640 1 -- 172.21.15.97:0/388262628 wait complete. 2024-03-29T04:08:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.562+0000 7f6c4584a640 1 Processor -- start 2024-03-29T04:08:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.563+0000 7f6c4584a640 1 -- start start 2024-03-29T04:08:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.563+0000 7f6c4584a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c4009bc60 0x7f6c400cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.563+0000 7f6c4584a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c400d0030 0x7f6c40147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.563+0000 7f6c4584a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4015c7e0 0x7f6c400c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.563+0000 7f6c4584a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c400701b0 con 0x7f6c4015c7e0 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.563+0000 7f6c4584a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40070030 con 0x7f6c4009bc60 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.563+0000 7f6c4584a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f6c40070330 con 0x7f6c400d0030 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.563+0000 7f6c3f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4015c7e0 0x7f6c400c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.563+0000 7f6c3f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4015c7e0 0x7f6c400c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50152/0 (socket says 172.21.15.97:50152) 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.563+0000 7f6c3f7fe640 1 -- 172.21.15.97:0/3690322191 learned_addr learned my addr 172.21.15.97:0/3690322191 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.564+0000 7f6c3effd640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c4009bc60 0x7f6c400cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.564+0000 7f6c3e7fc640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c400d0030 0x7f6c40147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.564+0000 7f6c3f7fe640 1 -- 172.21.15.97:0/3690322191 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c400d0030 msgr2=0x7f6c40147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.564+0000 7f6c3f7fe640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c400d0030 0x7f6c40147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.564+0000 7f6c3f7fe640 1 -- 172.21.15.97:0/3690322191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c4009bc60 msgr2=0x7f6c400cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.564+0000 7f6c3f7fe640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c4009bc60 0x7f6c400cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.564+0000 7f6c3f7fe640 1 -- 172.21.15.97:0/3690322191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4006c870 con 0x7f6c4015c7e0 2024-03-29T04:08:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.564+0000 7f6c3f7fe640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4015c7e0 0x7f6c400c8480 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800c3c0 tx=0x7f6c3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.564+0000 7f6c27fff640 1 -- 172.21.15.97:0/3690322191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c380052d0 con 0x7f6c4015c7e0 2024-03-29T04:08:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.564+0000 7f6c27fff640 1 -- 172.21.15.97:0/3690322191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c38005490 con 0x7f6c4015c7e0 2024-03-29T04:08:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.564+0000 7f6c27fff640 1 -- 172.21.15.97:0/3690322191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38007260 con 0x7f6c4015c7e0 2024-03-29T04:08:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.564+0000 7f6c4584a640 1 -- 172.21.15.97:0/3690322191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4006cb50 con 0x7f6c4015c7e0 2024-03-29T04:08:26.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.564+0000 7f6c4584a640 1 -- 172.21.15.97:0/3690322191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c400c5ea0 con 0x7f6c4015c7e0 2024-03-29T04:08:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.565+0000 7f6c4584a640 1 -- 172.21.15.97:0/3690322191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c40068d50 con 0x7f6c4015c7e0 2024-03-29T04:08:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.566+0000 7f6c27fff640 1 -- 172.21.15.97:0/3690322191 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6c38007400 con 0x7f6c4015c7e0 2024-03-29T04:08:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.566+0000 7f6c27fff640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6c2c06c530 0x7f6c2c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.566+0000 7f6c3effd640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6c2c06c530 0x7f6c2c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.566+0000 7f6c27fff640 1 -- 172.21.15.97:0/3690322191 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f6c3808a690 con 0x7f6c4015c7e0 2024-03-29T04:08:26.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.566+0000 7f6c3effd640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6c2c06c530 0x7f6c2c06e9f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6c30035000 tx=0x7f6c30030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:26.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.570+0000 7f6c27fff640 1 -- 172.21.15.97:0/3690322191 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6c38058a40 con 0x7f6c4015c7e0 2024-03-29T04:08:26.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.814+0000 7f6c4584a640 1 -- 172.21.15.97:0/3690322191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f6c40071b10 con 0x7f6c4015c7e0 2024-03-29T04:08:26.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.816+0000 7f6c27fff640 1 -- 172.21.15.97:0/3690322191 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f6c3805cd40 con 0x7f6c4015c7e0 2024-03-29T04:08:26.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.819+0000 7f6c4584a640 1 -- 172.21.15.97:0/3690322191 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6c2c06c530 msgr2=0x7f6c2c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.819+0000 7f6c4584a640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6c2c06c530 0x7f6c2c06e9f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6c30035000 tx=0x7f6c30030000 comp rx=0 tx=0).stop 2024-03-29T04:08:26.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.819+0000 7f6c4584a640 1 -- 172.21.15.97:0/3690322191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4015c7e0 msgr2=0x7f6c400c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.819+0000 7f6c4584a640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4015c7e0 0x7f6c400c8480 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800c3c0 tx=0x7f6c3800c880 comp rx=0 tx=0).stop 2024-03-29T04:08:26.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.819+0000 7f6c4584a640 1 -- 172.21.15.97:0/3690322191 shutdown_connections 2024-03-29T04:08:26.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.819+0000 7f6c4584a640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f6c2c06c530 0x7f6c2c06e9f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.819+0000 7f6c4584a640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4015c7e0 0x7f6c400c8480 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.819+0000 7f6c4584a640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6c400d0030 0x7f6c40147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.819+0000 7f6c4584a640 1 --2- 172.21.15.97:0/3690322191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c4009bc60 0x7f6c400cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.819+0000 7f6c4584a640 1 -- 172.21.15.97:0/3690322191 >> 172.21.15.97:0/3690322191 conn(0x7f6c400d2460 msgr2=0x7f6c4015b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:26.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.819+0000 7f6c4584a640 1 -- 172.21.15.97:0/3690322191 shutdown_connections 2024-03-29T04:08:26.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.819+0000 7f6c4584a640 1 -- 172.21.15.97:0/3690322191 wait complete. 2024-03-29T04:08:26.809 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-29T04:08:26.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.931+0000 7f747c704640 1 Processor -- start 2024-03-29T04:08:26.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.931+0000 7f747c704640 1 -- start start 2024-03-29T04:08:26.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.932+0000 7f747c704640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7474161ef0 0x7f74740d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.932+0000 7f747c704640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74740cd680 con 0x7f74741622d0 2024-03-29T04:08:26.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.932+0000 7f747c704640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74740cd960 con 0x7f747409b140 2024-03-29T04:08:26.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.932+0000 7f747c704640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f74740ce720 con 0x7f7474161ef0 2024-03-29T04:08:26.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.932+0000 7f7479c9f640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f74741622d0 0x7f747409aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59154/0 (socket says 172.21.15.97:59154) 2024-03-29T04:08:26.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.932+0000 7f7479c9f640 1 -- 172.21.15.97:0/3988766206 learned_addr learned my addr 172.21.15.97:0/3988766206 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:26.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.932+0000 7f747a4a0640 1 --2- 172.21.15.97:0/3988766206 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7474161ef0 0x7f74740d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.933+0000 7f747949e640 1 -- 172.21.15.97:0/3988766206 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 889381601 0 0) 0x7f74740cd680 con 0x7f74741622d0 2024-03-29T04:08:26.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.933+0000 7f747949e640 1 -- 172.21.15.97:0/3988766206 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7460003520 con 0x7f74741622d0 2024-03-29T04:08:26.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.933+0000 7f747949e640 1 -- 172.21.15.97:0/3988766206 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3161226412 0 0) 0x7f74740cd960 con 0x7f747409b140 2024-03-29T04:08:26.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.933+0000 7f747949e640 1 -- 172.21.15.97:0/3988766206 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74740cd680 con 0x7f747409b140 2024-03-29T04:08:26.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.933+0000 7f747a4a0640 1 -- 172.21.15.97:0/3988766206 >> v1:172.21.15.154:6789/0 conn(0x7f747409b140 legacy=0x7f747406c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.933+0000 7f747a4a0640 1 -- 172.21.15.97:0/3988766206 >> v1:172.21.15.97:6789/0 conn(0x7f74741622d0 legacy=0x7f747409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.933+0000 7f747a4a0640 1 -- 172.21.15.97:0/3988766206 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74740cdc40 con 0x7f7474161ef0 2024-03-29T04:08:26.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.933+0000 7f747a4a0640 1 --2- 172.21.15.97:0/3988766206 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7474161ef0 0x7f74740d0f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f74700097d0 tx=0x7f747002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4d8529a58cf64cb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:26.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.934+0000 7f747949e640 1 -- 172.21.15.97:0/3988766206 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470035020 con 0x7f7474161ef0 2024-03-29T04:08:26.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.934+0000 7f747949e640 1 -- 172.21.15.97:0/3988766206 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7470031800 con 0x7f7474161ef0 2024-03-29T04:08:26.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.934+0000 7f747949e640 1 -- 172.21.15.97:0/3988766206 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470031bd0 con 0x7f7474161ef0 2024-03-29T04:08:26.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.934+0000 7f747c704640 1 -- 172.21.15.97:0/3988766206 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7474161ef0 msgr2=0x7f74740d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.934+0000 7f747c704640 1 --2- 172.21.15.97:0/3988766206 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7474161ef0 0x7f74740d0f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f74700097d0 tx=0x7f747002b8a0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.934+0000 7f747c704640 1 -- 172.21.15.97:0/3988766206 shutdown_connections 2024-03-29T04:08:26.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.934+0000 7f747c704640 1 --2- 172.21.15.97:0/3988766206 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7474161ef0 0x7f74740d0f30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.934+0000 7f747c704640 1 -- 172.21.15.97:0/3988766206 >> 172.21.15.97:0/3988766206 conn(0x7f74740b3f00 msgr2=0x7f74740b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:26.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.934+0000 7f747c704640 1 -- 172.21.15.97:0/3988766206 shutdown_connections 2024-03-29T04:08:26.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.934+0000 7f747c704640 1 -- 172.21.15.97:0/3988766206 wait complete. 2024-03-29T04:08:26.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.935+0000 7f747c704640 1 Processor -- start 2024-03-29T04:08:26.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.935+0000 7f747c704640 1 -- start start 2024-03-29T04:08:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.936+0000 7f747c704640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747409b140 0x7f7474158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.936+0000 7f747c704640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7474161ef0 0x7f74740c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.936+0000 7f747c704640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74741622d0 0x7f74741f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.936+0000 7f747c704640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f74740cd300 con 0x7f747409b140 2024-03-29T04:08:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.936+0000 7f747c704640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f74740ce290 con 0x7f7474161ef0 2024-03-29T04:08:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.936+0000 7f747c704640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f74740d1470 con 0x7f74741622d0 2024-03-29T04:08:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.936+0000 7f747a4a0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747409b140 0x7f7474158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.936+0000 7f747a4a0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747409b140 0x7f7474158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50160/0 (socket says 172.21.15.97:50160) 2024-03-29T04:08:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.936+0000 7f747a4a0640 1 -- 172.21.15.97:0/2577941812 learned_addr learned my addr 172.21.15.97:0/2577941812 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.936+0000 7f7479c9f640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7474161ef0 0x7f74740c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.936+0000 7f747aca1640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74741622d0 0x7f74741f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.936+0000 7f747a4a0640 1 -- 172.21.15.97:0/2577941812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74741622d0 msgr2=0x7f74741f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.937+0000 7f747a4a0640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74741622d0 0x7f74741f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.937+0000 7f747a4a0640 1 -- 172.21.15.97:0/2577941812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7474161ef0 msgr2=0x7f74740c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.937+0000 7f747a4a0640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7474161ef0 0x7f74740c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.937+0000 7f747a4a0640 1 -- 172.21.15.97:0/2577941812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74741591e0 con 0x7f747409b140 2024-03-29T04:08:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.937+0000 7f747aca1640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74741622d0 0x7f74741f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.937+0000 7f7479c9f640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7474161ef0 0x7f74740c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.937+0000 7f747a4a0640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747409b140 0x7f7474158ca0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f74700098f0 tx=0x7f74700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.937+0000 7f746affd640 1 -- 172.21.15.97:0/2577941812 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470041070 con 0x7f747409b140 2024-03-29T04:08:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.937+0000 7f746affd640 1 -- 172.21.15.97:0/2577941812 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7470031e30 con 0x7f747409b140 2024-03-29T04:08:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.937+0000 7f747c704640 1 -- 172.21.15.97:0/2577941812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74741f5380 con 0x7f747409b140 2024-03-29T04:08:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.937+0000 7f746affd640 1 -- 172.21.15.97:0/2577941812 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470035040 con 0x7f747409b140 2024-03-29T04:08:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.937+0000 7f747c704640 1 -- 172.21.15.97:0/2577941812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74740c9580 con 0x7f747409b140 2024-03-29T04:08:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.938+0000 7f747c704640 1 -- 172.21.15.97:0/2577941812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7474068640 con 0x7f747409b140 2024-03-29T04:08:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.938+0000 7f746affd640 1 -- 172.21.15.97:0/2577941812 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f747004c060 con 0x7f747409b140 2024-03-29T04:08:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.939+0000 7f746affd640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f745806c530 0x7f745806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.939+0000 7f746affd640 1 -- 172.21.15.97:0/2577941812 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f74700acda0 con 0x7f747409b140 2024-03-29T04:08:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.939+0000 7f7479c9f640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f745806c530 0x7f745806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.939+0000 7f7479c9f640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f745806c530 0x7f745806e9f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f74741595c0 tx=0x7f7464009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:26.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:26.944+0000 7f746affd640 1 -- 172.21.15.97:0/2577941812 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f747004b2c0 con 0x7f747409b140 2024-03-29T04:08:27.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.186+0000 7f747c704640 1 -- 172.21.15.97:0/2577941812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f74740b8390 con 0x7f747409b140 2024-03-29T04:08:27.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.188+0000 7f746affd640 1 -- 172.21.15.97:0/2577941812 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f747007b7a0 con 0x7f747409b140 2024-03-29T04:08:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.190+0000 7f747c704640 1 -- 172.21.15.97:0/2577941812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f745806c530 msgr2=0x7f745806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.190+0000 7f747c704640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f745806c530 0x7f745806e9f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f74741595c0 tx=0x7f7464009040 comp rx=0 tx=0).stop 2024-03-29T04:08:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.190+0000 7f747c704640 1 -- 172.21.15.97:0/2577941812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747409b140 msgr2=0x7f7474158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.190+0000 7f747c704640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747409b140 0x7f7474158ca0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f74700098f0 tx=0x7f74700046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.190+0000 7f747c704640 1 -- 172.21.15.97:0/2577941812 shutdown_connections 2024-03-29T04:08:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.190+0000 7f747c704640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f745806c530 0x7f745806e9f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.190+0000 7f747c704640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74741622d0 0x7f74741f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.190+0000 7f747c704640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7474161ef0 0x7f74740c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.190+0000 7f747c704640 1 --2- 172.21.15.97:0/2577941812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747409b140 0x7f7474158ca0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.190+0000 7f747c704640 1 -- 172.21.15.97:0/2577941812 >> 172.21.15.97:0/2577941812 conn(0x7f74740b3f00 msgr2=0x7f74740b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:27.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.190+0000 7f747c704640 1 -- 172.21.15.97:0/2577941812 shutdown_connections 2024-03-29T04:08:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.190+0000 7f747c704640 1 -- 172.21.15.97:0/2577941812 wait complete. 2024-03-29T04:08:27.180 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:27.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.300+0000 7f3e0cd52640 1 Processor -- start 2024-03-29T04:08:27.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.300+0000 7f3e0cd52640 1 -- start start 2024-03-29T04:08:27.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.301+0000 7f3e0cd52640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3e080b1430 0x7f3e080b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.301+0000 7f3e0cd52640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e0815d4b0 0x7f3e080cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.301+0000 7f3e0cd52640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e0806b880 con 0x7f3e08160300 2024-03-29T04:08:27.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.301+0000 7f3e0cd52640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f3e0806c490 con 0x7f3e0815d4b0 2024-03-29T04:08:27.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.301+0000 7f3e0cd52640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f3e0806bb60 con 0x7f3e080b1430 2024-03-29T04:08:27.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.301+0000 7f3e06d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e0815d4b0 0x7f3e080cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.301+0000 7f3e06d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e0815d4b0 0x7f3e080cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50162/0 (socket says 172.21.15.97:50162) 2024-03-29T04:08:27.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.301+0000 7f3e06d9d640 1 -- 172.21.15.97:0/88518925 learned_addr learned my addr 172.21.15.97:0/88518925 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:27.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.302+0000 7f3e0659c640 1 --2- 172.21.15.97:0/88518925 >> v2:172.21.15.154:3300/0 conn(0x7f3e080b1430 0x7f3e080b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.302+0000 7f3e06d9d640 1 -- 172.21.15.97:0/88518925 >> v2:172.21.15.154:3300/0 conn(0x7f3e080b1430 msgr2=0x7f3e080b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:27.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.302+0000 7f3e06d9d640 1 --2- 172.21.15.97:0/88518925 >> v2:172.21.15.154:3300/0 conn(0x7f3e080b1430 0x7f3e080b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.302+0000 7f3e06d9d640 1 -- 172.21.15.97:0/88518925 >> v1:172.21.15.154:6789/0 conn(0x7f3e08160300 legacy=0x7f3e0815cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.302+0000 7f3e06d9d640 1 -- 172.21.15.97:0/88518925 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0806be40 con 0x7f3e0815d4b0 2024-03-29T04:08:27.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.302+0000 7f3e06d9d640 1 --2- 172.21.15.97:0/88518925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0815d4b0 0x7f3e080cdff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3e080692b0 tx=0x7f3e0002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8c752c3490055a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:27.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.303+0000 7f3e0559a640 1 -- 172.21.15.97:0/88518925 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0002cb20 con 0x7f3e0815d4b0 2024-03-29T04:08:27.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.303+0000 7f3e0559a640 1 -- 172.21.15.97:0/88518925 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0002ccc0 con 0x7f3e0815d4b0 2024-03-29T04:08:27.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.303+0000 7f3e0559a640 1 -- 172.21.15.97:0/88518925 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00034610 con 0x7f3e0815d4b0 2024-03-29T04:08:27.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.303+0000 7f3e0cd52640 1 -- 172.21.15.97:0/88518925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0815d4b0 msgr2=0x7f3e080cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.303+0000 7f3e0cd52640 1 --2- 172.21.15.97:0/88518925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0815d4b0 0x7f3e080cdff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3e080692b0 tx=0x7f3e0002bf80 comp rx=0 tx=0).stop 2024-03-29T04:08:27.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.303+0000 7f3e0cd52640 1 -- 172.21.15.97:0/88518925 shutdown_connections 2024-03-29T04:08:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.303+0000 7f3e0cd52640 1 --2- 172.21.15.97:0/88518925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0815d4b0 0x7f3e080cdff0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.303+0000 7f3e0cd52640 1 --2- 172.21.15.97:0/88518925 >> v2:172.21.15.154:3300/0 conn(0x7f3e080b1430 0x7f3e080b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.303+0000 7f3e0cd52640 1 -- 172.21.15.97:0/88518925 >> 172.21.15.97:0/88518925 conn(0x7f3e080b6860 msgr2=0x7f3e080b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.303+0000 7f3e0cd52640 1 -- 172.21.15.97:0/88518925 shutdown_connections 2024-03-29T04:08:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.303+0000 7f3e0cd52640 1 -- 172.21.15.97:0/88518925 wait complete. 2024-03-29T04:08:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.304+0000 7f3e0cd52640 1 Processor -- start 2024-03-29T04:08:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.305+0000 7f3e0cd52640 1 -- start start 2024-03-29T04:08:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.305+0000 7f3e0cd52640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e080b1430 0x7f3e081547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.305+0000 7f3e0cd52640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0815d4b0 0x7f3e0814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.305+0000 7f3e0cd52640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e08160300 0x7f3e080ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.305+0000 7f3e0cd52640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0806b510 con 0x7f3e0815d4b0 2024-03-29T04:08:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.305+0000 7f3e0cd52640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3e080734f0 con 0x7f3e080b1430 2024-03-29T04:08:27.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.305+0000 7f3e0cd52640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3e080b1d50 con 0x7f3e08160300 2024-03-29T04:08:27.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.305+0000 7f3e05d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0815d4b0 0x7f3e0814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.305+0000 7f3e05d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0815d4b0 0x7f3e0814fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50176/0 (socket says 172.21.15.97:50176) 2024-03-29T04:08:27.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.305+0000 7f3e05d9b640 1 -- 172.21.15.97:0/95638366 learned_addr learned my addr 172.21.15.97:0/95638366 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:27.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.305+0000 7f3e0659c640 1 --2- 172.21.15.97:0/95638366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e080b1430 0x7f3e081547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.306+0000 7f3e05d9b640 1 -- 172.21.15.97:0/95638366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e08160300 msgr2=0x7f3e080ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.306+0000 7f3e05d9b640 1 --2- 172.21.15.97:0/95638366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e08160300 0x7f3e080ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.306+0000 7f3e05d9b640 1 -- 172.21.15.97:0/95638366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e080b1430 msgr2=0x7f3e081547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.306+0000 7f3e05d9b640 1 --2- 172.21.15.97:0/95638366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e080b1430 0x7f3e081547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.306+0000 7f3e05d9b640 1 -- 172.21.15.97:0/95638366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e08154130 con 0x7f3e0815d4b0 2024-03-29T04:08:27.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.306+0000 7f3e05d9b640 1 --2- 172.21.15.97:0/95638366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0815d4b0 0x7f3e0814fc70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00c450 tx=0x7f3dfc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:27.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.306+0000 7f3df6ffd640 1 -- 172.21.15.97:0/95638366 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0052d0 con 0x7f3e0815d4b0 2024-03-29T04:08:27.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.306+0000 7f3df6ffd640 1 -- 172.21.15.97:0/95638366 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dfc005470 con 0x7f3e0815d4b0 2024-03-29T04:08:27.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.306+0000 7f3e0cd52640 1 -- 172.21.15.97:0/95638366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e081f57d0 con 0x7f3e0815d4b0 2024-03-29T04:08:27.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.306+0000 7f3df6ffd640 1 -- 172.21.15.97:0/95638366 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc012870 con 0x7f3e0815d4b0 2024-03-29T04:08:27.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.306+0000 7f3e0cd52640 1 -- 172.21.15.97:0/95638366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e081f5a90 con 0x7f3e0815d4b0 2024-03-29T04:08:27.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.307+0000 7f3e0cd52640 1 -- 172.21.15.97:0/95638366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e08068640 con 0x7f3e0815d4b0 2024-03-29T04:08:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.307+0000 7f3df6ffd640 1 -- 172.21.15.97:0/95638366 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3dfc012a10 con 0x7f3e0815d4b0 2024-03-29T04:08:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.308+0000 7f3df6ffd640 1 --2- 172.21.15.97:0/95638366 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3de806c530 0x7f3de806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.308+0000 7f3df6ffd640 1 -- 172.21.15.97:0/95638366 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f3dfc08ae90 con 0x7f3e0815d4b0 2024-03-29T04:08:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.308+0000 7f3e0659c640 1 --2- 172.21.15.97:0/95638366 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3de806c530 0x7f3de806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.308+0000 7f3e0659c640 1 --2- 172.21.15.97:0/95638366 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3de806c530 0x7f3de806e9f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3df80079a0 tx=0x7f3df8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:27.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.312+0000 7f3df6ffd640 1 -- 172.21.15.97:0/95638366 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3dfc059800 con 0x7f3e0815d4b0 2024-03-29T04:08:27.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.491+0000 7f3e0cd52640 1 -- 172.21.15.97:0/95638366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e080d1640 con 0x7f3e0815d4b0 2024-03-29T04:08:27.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.492+0000 7f3df6ffd640 1 -- 172.21.15.97:0/95638366 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+12891 (secure 0 0 0) 0x7f3dfc059100 con 0x7f3e0815d4b0 2024-03-29T04:08:27.471 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:27.471 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:08:24.598399+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:27.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.494+0000 7f3e0cd52640 1 -- 172.21.15.97:0/95638366 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3de806c530 msgr2=0x7f3de806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.494+0000 7f3e0cd52640 1 --2- 172.21.15.97:0/95638366 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3de806c530 0x7f3de806e9f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3df80079a0 tx=0x7f3df8008040 comp rx=0 tx=0).stop 2024-03-29T04:08:27.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.494+0000 7f3e0cd52640 1 -- 172.21.15.97:0/95638366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0815d4b0 msgr2=0x7f3e0814fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.494+0000 7f3e0cd52640 1 --2- 172.21.15.97:0/95638366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0815d4b0 0x7f3e0814fc70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00c450 tx=0x7f3dfc00c910 comp rx=0 tx=0).stop 2024-03-29T04:08:27.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.494+0000 7f3e0cd52640 1 -- 172.21.15.97:0/95638366 shutdown_connections 2024-03-29T04:08:27.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.494+0000 7f3e0cd52640 1 --2- 172.21.15.97:0/95638366 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3de806c530 0x7f3de806e9f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.494+0000 7f3e0cd52640 1 --2- 172.21.15.97:0/95638366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e08160300 0x7f3e080ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.494+0000 7f3e0cd52640 1 --2- 172.21.15.97:0/95638366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e0815d4b0 0x7f3e0814fc70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.494+0000 7f3e0cd52640 1 --2- 172.21.15.97:0/95638366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e080b1430 0x7f3e081547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.494+0000 7f3e0cd52640 1 -- 172.21.15.97:0/95638366 >> 172.21.15.97:0/95638366 conn(0x7f3e080b6860 msgr2=0x7f3e080d1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:27.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.494+0000 7f3e0cd52640 1 -- 172.21.15.97:0/95638366 shutdown_connections 2024-03-29T04:08:27.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.494+0000 7f3e0cd52640 1 -- 172.21.15.97:0/95638366 wait complete. 2024-03-29T04:08:27.485 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:08:27.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:08:27.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.611+0000 7f59e7ace640 1 Processor -- start 2024-03-29T04:08:27.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.612+0000 7f59e7ace640 1 -- start start 2024-03-29T04:08:27.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.612+0000 7f59e7ace640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f59e015c760 0x7f59e00d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.612+0000 7f59e7ace640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f59e009b640 0x7f59e009bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.612+0000 7f59e7ace640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59e00d11d0 con 0x7f59e00af2b0 2024-03-29T04:08:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.612+0000 7f59e7ace640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f59e009ab60 con 0x7f59e015c760 2024-03-29T04:08:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.612+0000 7f59e7ace640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f59e009ae40 con 0x7f59e009b640 2024-03-29T04:08:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.612+0000 7f59e586a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f59e015c760 0x7f59e00d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.612+0000 7f59e586a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f59e015c760 0x7f59e00d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50186/0 (socket says 172.21.15.97:50186) 2024-03-29T04:08:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.612+0000 7f59e586a640 1 -- 172.21.15.97:0/2969786754 learned_addr learned my addr 172.21.15.97:0/2969786754 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.613+0000 7f59e586a640 1 -- 172.21.15.97:0/2969786754 >> v2:172.21.15.154:3300/0 conn(0x7f59e009b640 msgr2=0x7f59e009bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.613+0000 7f59e586a640 1 --2- 172.21.15.97:0/2969786754 >> v2:172.21.15.154:3300/0 conn(0x7f59e009b640 0x7f59e009bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.613+0000 7f59e586a640 1 -- 172.21.15.97:0/2969786754 >> v1:172.21.15.154:6789/0 conn(0x7f59e00af2b0 legacy=0x7f59e0160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.613+0000 7f59e586a640 1 -- 172.21.15.97:0/2969786754 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e009b120 con 0x7f59e015c760 2024-03-29T04:08:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.613+0000 7f59e586a640 1 --2- 172.21.15.97:0/2969786754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59e015c760 0x7f59e00d0aa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f59e006bf20 tx=0x7f59dc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452d7c33c5b58ef1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.614+0000 7f59e4868640 1 -- 172.21.15.97:0/2969786754 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc02c9f0 con 0x7f59e015c760 2024-03-29T04:08:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.614+0000 7f59e4868640 1 -- 172.21.15.97:0/2969786754 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59dc02cb90 con 0x7f59e015c760 2024-03-29T04:08:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.614+0000 7f59e4868640 1 -- 172.21.15.97:0/2969786754 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0343f0 con 0x7f59e015c760 2024-03-29T04:08:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.614+0000 7f59e7ace640 1 -- 172.21.15.97:0/2969786754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59e015c760 msgr2=0x7f59e00d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.614+0000 7f59e7ace640 1 --2- 172.21.15.97:0/2969786754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59e015c760 0x7f59e00d0aa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f59e006bf20 tx=0x7f59dc02be50 comp rx=0 tx=0).stop 2024-03-29T04:08:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.614+0000 7f59e7ace640 1 -- 172.21.15.97:0/2969786754 shutdown_connections 2024-03-29T04:08:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.614+0000 7f59e7ace640 1 --2- 172.21.15.97:0/2969786754 >> v2:172.21.15.154:3300/0 conn(0x7f59e009b640 0x7f59e009bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.614+0000 7f59e7ace640 1 --2- 172.21.15.97:0/2969786754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59e015c760 0x7f59e00d0aa0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.614+0000 7f59e7ace640 1 -- 172.21.15.97:0/2969786754 >> 172.21.15.97:0/2969786754 conn(0x7f59e00d24c0 msgr2=0x7f59e00d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.614+0000 7f59e7ace640 1 -- 172.21.15.97:0/2969786754 shutdown_connections 2024-03-29T04:08:27.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.614+0000 7f59e7ace640 1 -- 172.21.15.97:0/2969786754 wait complete. 2024-03-29T04:08:27.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.615+0000 7f59e7ace640 1 Processor -- start 2024-03-29T04:08:27.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.615+0000 7f59e7ace640 1 -- start start 2024-03-29T04:08:27.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e7ace640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59e009b640 0x7f59e00c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e7ace640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59e00af2b0 0x7f59e015ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e7ace640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59e015c760 0x7f59e0151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e7ace640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f59e015ccc0 con 0x7f59e009b640 2024-03-29T04:08:27.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e7ace640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f59e015cb40 con 0x7f59e00af2b0 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e7ace640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f59e00af6c0 con 0x7f59e015c760 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e586a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59e009b640 0x7f59e00c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e586a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59e009b640 0x7f59e00c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50194/0 (socket says 172.21.15.97:50194) 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e586a640 1 -- 172.21.15.97:0/3222867107 learned_addr learned my addr 172.21.15.97:0/3222867107 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e586a640 1 -- 172.21.15.97:0/3222867107 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59e015c760 msgr2=0x7f59e0151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e586a640 1 --2- 172.21.15.97:0/3222867107 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59e015c760 0x7f59e0151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e586a640 1 -- 172.21.15.97:0/3222867107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59e00af2b0 msgr2=0x7f59e015ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e586a640 1 --2- 172.21.15.97:0/3222867107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59e00af2b0 0x7f59e015ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.616+0000 7f59e586a640 1 -- 172.21.15.97:0/3222867107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e015d590 con 0x7f59e009b640 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.617+0000 7f59e586a640 1 --2- 172.21.15.97:0/3222867107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59e009b640 0x7f59e00c7170 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0093e0 tx=0x7f59dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.617+0000 7f59d67fc640 1 -- 172.21.15.97:0/3222867107 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc043020 con 0x7f59e009b640 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.617+0000 7f59d67fc640 1 -- 172.21.15.97:0/3222867107 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59dc03e450 con 0x7f59e009b640 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.617+0000 7f59e7ace640 1 -- 172.21.15.97:0/3222867107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e015d870 con 0x7f59e009b640 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.617+0000 7f59d67fc640 1 -- 172.21.15.97:0/3222867107 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc03e840 con 0x7f59e009b640 2024-03-29T04:08:27.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.617+0000 7f59e7ace640 1 -- 172.21.15.97:0/3222867107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e015de30 con 0x7f59e009b640 2024-03-29T04:08:27.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.618+0000 7f59e7ace640 1 -- 172.21.15.97:0/3222867107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e00a0630 con 0x7f59e009b640 2024-03-29T04:08:27.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.618+0000 7f59d67fc640 1 -- 172.21.15.97:0/3222867107 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f59dc046020 con 0x7f59e009b640 2024-03-29T04:08:27.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.619+0000 7f59d67fc640 1 --2- 172.21.15.97:0/3222867107 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f59c406c560 0x7f59c406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.619+0000 7f59e5069640 1 --2- 172.21.15.97:0/3222867107 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f59c406c560 0x7f59c406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.619+0000 7f59d67fc640 1 -- 172.21.15.97:0/3222867107 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f59dc0ada50 con 0x7f59e009b640 2024-03-29T04:08:27.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.619+0000 7f59e5069640 1 --2- 172.21.15.97:0/3222867107 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f59c406c560 0x7f59c406ea20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f59e00c7960 tx=0x7f59d0006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:27.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.623+0000 7f59d67fc640 1 -- 172.21.15.97:0/3222867107 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f59dc04d070 con 0x7f59e009b640 2024-03-29T04:08:27.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.796+0000 7f59e7ace640 1 -- 172.21.15.97:0/3222867107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f59e004f9d0 con 0x7f59e009b640 2024-03-29T04:08:27.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.796+0000 7f59d67fc640 1 -- 172.21.15.97:0/3222867107 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f59dc07c450 con 0x7f59e009b640 2024-03-29T04:08:27.776 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:08:27.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.798+0000 7f59e7ace640 1 -- 172.21.15.97:0/3222867107 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f59c406c560 msgr2=0x7f59c406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.798+0000 7f59e7ace640 1 --2- 172.21.15.97:0/3222867107 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f59c406c560 0x7f59c406ea20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f59e00c7960 tx=0x7f59d0006c90 comp rx=0 tx=0).stop 2024-03-29T04:08:27.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.798+0000 7f59e7ace640 1 -- 172.21.15.97:0/3222867107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59e009b640 msgr2=0x7f59e00c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.798+0000 7f59e7ace640 1 --2- 172.21.15.97:0/3222867107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59e009b640 0x7f59e00c7170 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0093e0 tx=0x7f59dc004a40 comp rx=0 tx=0).stop 2024-03-29T04:08:27.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.798+0000 7f59e7ace640 1 -- 172.21.15.97:0/3222867107 shutdown_connections 2024-03-29T04:08:27.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.798+0000 7f59e7ace640 1 --2- 172.21.15.97:0/3222867107 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f59c406c560 0x7f59c406ea20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.798+0000 7f59e7ace640 1 --2- 172.21.15.97:0/3222867107 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59e015c760 0x7f59e0151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.798+0000 7f59e7ace640 1 --2- 172.21.15.97:0/3222867107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59e00af2b0 0x7f59e015ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.798+0000 7f59e7ace640 1 --2- 172.21.15.97:0/3222867107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59e009b640 0x7f59e00c7170 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.798+0000 7f59e7ace640 1 -- 172.21.15.97:0/3222867107 >> 172.21.15.97:0/3222867107 conn(0x7f59e00d24c0 msgr2=0x7f59e00b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:27.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.798+0000 7f59e7ace640 1 -- 172.21.15.97:0/3222867107 shutdown_connections 2024-03-29T04:08:27.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.798+0000 7f59e7ace640 1 -- 172.21.15.97:0/3222867107 wait complete. 2024-03-29T04:08:27.788 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T04:08:27.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.909+0000 7f6273b04640 1 Processor -- start 2024-03-29T04:08:27.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.909+0000 7f6273b04640 1 -- start start 2024-03-29T04:08:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f6273b04640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f626c0d0020 0x7f626c09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f6273b04640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f626c0b1a40 0x7f626c09af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f6273b04640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f626c0d04b0 con 0x7f626c0b1660 2024-03-29T04:08:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f6273b04640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f626c0ce4a0 con 0x7f626c0d0020 2024-03-29T04:08:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f6273b04640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f626c0d0790 con 0x7f626c0b1a40 2024-03-29T04:08:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f627109f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f626c0d0020 0x7f626c09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f627109f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f626c0d0020 0x7f626c09aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50206/0 (socket says 172.21.15.97:50206) 2024-03-29T04:08:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f627109f640 1 -- 172.21.15.97:0/887600276 learned_addr learned my addr 172.21.15.97:0/887600276 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f62720a1640 1 --2- 172.21.15.97:0/887600276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f626c0b1a40 0x7f626c09af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f627109f640 1 -- 172.21.15.97:0/887600276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f626c0b1a40 msgr2=0x7f626c09af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f627109f640 1 --2- 172.21.15.97:0/887600276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f626c0b1a40 0x7f626c09af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f627109f640 1 -- 172.21.15.97:0/887600276 >> v1:172.21.15.154:6789/0 conn(0x7f626c0b1660 legacy=0x7f626c0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f627109f640 1 -- 172.21.15.97:0/887600276 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c0d0a70 con 0x7f626c0d0020 2024-03-29T04:08:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.910+0000 7f62720a1640 1 --2- 172.21.15.97:0/887600276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f626c0b1a40 0x7f626c09af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.911+0000 7f627109f640 1 --2- 172.21.15.97:0/887600276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c0d0020 0x7f626c09aa30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f625c009870 tx=0x7f625c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ccf52890b2aa7dd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.911+0000 7f627089e640 1 -- 172.21.15.97:0/887600276 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c035020 con 0x7f626c0d0020 2024-03-29T04:08:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.911+0000 7f627089e640 1 -- 172.21.15.97:0/887600276 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f625c031640 con 0x7f626c0d0020 2024-03-29T04:08:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.911+0000 7f627089e640 1 -- 172.21.15.97:0/887600276 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c02d040 con 0x7f626c0d0020 2024-03-29T04:08:27.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.911+0000 7f6273b04640 1 -- 172.21.15.97:0/887600276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c0d0020 msgr2=0x7f626c09aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.911+0000 7f6273b04640 1 --2- 172.21.15.97:0/887600276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c0d0020 0x7f626c09aa30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f625c009870 tx=0x7f625c02b8f0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.912+0000 7f6273b04640 1 -- 172.21.15.97:0/887600276 shutdown_connections 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.912+0000 7f6273b04640 1 --2- 172.21.15.97:0/887600276 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f626c0b1a40 0x7f626c09af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.912+0000 7f6273b04640 1 --2- 172.21.15.97:0/887600276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c0d0020 0x7f626c09aa30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.912+0000 7f6273b04640 1 -- 172.21.15.97:0/887600276 >> 172.21.15.97:0/887600276 conn(0x7f626c0b3f40 msgr2=0x7f626c0b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.912+0000 7f6273b04640 1 -- 172.21.15.97:0/887600276 shutdown_connections 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.912+0000 7f6273b04640 1 -- 172.21.15.97:0/887600276 wait complete. 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.913+0000 7f6273b04640 1 Processor -- start 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.913+0000 7f6273b04640 1 -- start start 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.913+0000 7f6273b04640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f626c0b1660 0x7f626c158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.913+0000 7f6273b04640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f626c0b1a40 0x7f626c0c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.913+0000 7f6273b04640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c0c75a0 0x7f626c15e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.913+0000 7f6273b04640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f626c15e0c0 con 0x7f626c0c75a0 2024-03-29T04:08:27.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.913+0000 7f6273b04640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f626c0d1030 con 0x7f626c0b1a40 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.913+0000 7f6273b04640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f626c15e240 con 0x7f626c0b1660 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.913+0000 7f62720a1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c0c75a0 0x7f626c15e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f62720a1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c0c75a0 0x7f626c15e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50220/0 (socket says 172.21.15.97:50220) 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f62720a1640 1 -- 172.21.15.97:0/4233392932 learned_addr learned my addr 172.21.15.97:0/4233392932 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f627109f640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f626c0b1a40 0x7f626c0c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f62718a0640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f626c0b1660 0x7f626c158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f62720a1640 1 -- 172.21.15.97:0/4233392932 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f626c0b1660 msgr2=0x7f626c158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f62720a1640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f626c0b1660 0x7f626c158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f62720a1640 1 -- 172.21.15.97:0/4233392932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f626c0b1a40 msgr2=0x7f626c0c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f62720a1640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f626c0b1a40 0x7f626c0c7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f62720a1640 1 -- 172.21.15.97:0/4233392932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c0c7a40 con 0x7f626c0c75a0 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f627109f640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f626c0b1a40 0x7f626c0c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f62720a1640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c0c75a0 0x7f626c15e600 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f626400cef0 tx=0x7f626400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f62627fc640 1 -- 172.21.15.97:0/4233392932 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626401a070 con 0x7f626c0c75a0 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f62627fc640 1 -- 172.21.15.97:0/4233392932 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f626400d860 con 0x7f626c0c75a0 2024-03-29T04:08:27.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f62627fc640 1 -- 172.21.15.97:0/4233392932 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626400dc50 con 0x7f626c0c75a0 2024-03-29T04:08:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f6273b04640 1 -- 172.21.15.97:0/4233392932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626c0c7d20 con 0x7f626c0c75a0 2024-03-29T04:08:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.914+0000 7f6273b04640 1 -- 172.21.15.97:0/4233392932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626c0c85c0 con 0x7f626c0c75a0 2024-03-29T04:08:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.915+0000 7f6273b04640 1 -- 172.21.15.97:0/4233392932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626c0c8110 con 0x7f626c0c75a0 2024-03-29T04:08:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.916+0000 7f62627fc640 1 -- 172.21.15.97:0/4233392932 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f626401d070 con 0x7f626c0c75a0 2024-03-29T04:08:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.916+0000 7f62627fc640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f625006c530 0x7f625006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.916+0000 7f62627fc640 1 -- 172.21.15.97:0/4233392932 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f626408bd10 con 0x7f626c0c75a0 2024-03-29T04:08:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.916+0000 7f62718a0640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f625006c530 0x7f625006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.917+0000 7f62718a0640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f625006c530 0x7f625006e9f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6268003d90 tx=0x7f6268009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:27.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:27.921+0000 7f62627fc640 1 -- 172.21.15.97:0/4233392932 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f626405a6b0 con 0x7f626c0c75a0 2024-03-29T04:08:28.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.095+0000 7f6273b04640 1 -- 172.21.15.97:0/4233392932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f626c068640 con 0x7f626c0c75a0 2024-03-29T04:08:28.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.095+0000 7f62627fc640 1 -- 172.21.15.97:0/4233392932 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) ==== 84+0+65 (secure 0 0 0) 0x7f6264059fb0 con 0x7f626c0c75a0 2024-03-29T04:08:28.075 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:28.075 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-03-29T04:08:28.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-03-29T04:08:28.076 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-03-29T04:08:28.076 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-03-29T04:08:28.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:28.076 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-03-29T04:08:28.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.097+0000 7f6273b04640 1 -- 172.21.15.97:0/4233392932 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f625006c530 msgr2=0x7f625006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.097+0000 7f6273b04640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f625006c530 0x7f625006e9f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6268003d90 tx=0x7f6268009040 comp rx=0 tx=0).stop 2024-03-29T04:08:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.097+0000 7f6273b04640 1 -- 172.21.15.97:0/4233392932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c0c75a0 msgr2=0x7f626c15e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.097+0000 7f6273b04640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c0c75a0 0x7f626c15e600 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f626400cef0 tx=0x7f626400a710 comp rx=0 tx=0).stop 2024-03-29T04:08:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.097+0000 7f6273b04640 1 -- 172.21.15.97:0/4233392932 shutdown_connections 2024-03-29T04:08:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.097+0000 7f6273b04640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f625006c530 0x7f625006e9f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.097+0000 7f6273b04640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c0c75a0 0x7f626c15e600 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.097+0000 7f6273b04640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f626c0b1a40 0x7f626c0c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.097+0000 7f6273b04640 1 --2- 172.21.15.97:0/4233392932 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f626c0b1660 0x7f626c158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.097+0000 7f6273b04640 1 -- 172.21.15.97:0/4233392932 >> 172.21.15.97:0/4233392932 conn(0x7f626c0b3f40 msgr2=0x7f626c0b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.097+0000 7f6273b04640 1 -- 172.21.15.97:0/4233392932 shutdown_connections 2024-03-29T04:08:28.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.097+0000 7f6273b04640 1 -- 172.21.15.97:0/4233392932 wait complete. 2024-03-29T04:08:28.088 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T04:08:28.088 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T04:08:28.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.213+0000 7f12ec5af640 1 Processor -- start 2024-03-29T04:08:28.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.213+0000 7f12ec5af640 1 -- start start 2024-03-29T04:08:28.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.214+0000 7f12ec5af640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f12e40cffe0 0x7f12e409b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.214+0000 7f12ec5af640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f12e409ba50 0x7f12e40cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.214+0000 7f12ec5af640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e409be80 con 0x7f12e415dc80 2024-03-29T04:08:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.214+0000 7f12ec5af640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f12e40cebf0 con 0x7f12e40cffe0 2024-03-29T04:08:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.214+0000 7f12ec5af640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f12e40714e0 con 0x7f12e409ba50 2024-03-29T04:08:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.214+0000 7f12e9b4a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f12e40cffe0 0x7f12e409b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.214+0000 7f12e9b4a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f12e40cffe0 0x7f12e409b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50222/0 (socket says 172.21.15.97:50222) 2024-03-29T04:08:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.214+0000 7f12e9b4a640 1 -- 172.21.15.97:0/2018554734 learned_addr learned my addr 172.21.15.97:0/2018554734 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.215+0000 7f12eab4c640 1 --2- 172.21.15.97:0/2018554734 >> v2:172.21.15.154:3300/0 conn(0x7f12e409ba50 0x7f12e40cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.215+0000 7f12e9b4a640 1 -- 172.21.15.97:0/2018554734 >> v2:172.21.15.154:3300/0 conn(0x7f12e409ba50 msgr2=0x7f12e40cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.215+0000 7f12e9b4a640 1 --2- 172.21.15.97:0/2018554734 >> v2:172.21.15.154:3300/0 conn(0x7f12e409ba50 0x7f12e40cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.215+0000 7f12e9b4a640 1 -- 172.21.15.97:0/2018554734 >> v1:172.21.15.154:6789/0 conn(0x7f12e415dc80 legacy=0x7f12e406fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:28.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.215+0000 7f12e9b4a640 1 -- 172.21.15.97:0/2018554734 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e40717c0 con 0x7f12e40cffe0 2024-03-29T04:08:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.215+0000 7f12e9b4a640 1 --2- 172.21.15.97:0/2018554734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12e40cffe0 0x7f12e409b510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f12e40d0770 tx=0x7f12d402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0049488995945e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.216+0000 7f12e9349640 1 -- 172.21.15.97:0/2018554734 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d402e070 con 0x7f12e40cffe0 2024-03-29T04:08:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.216+0000 7f12e9349640 1 -- 172.21.15.97:0/2018554734 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d4037070 con 0x7f12e40cffe0 2024-03-29T04:08:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.216+0000 7f12e9349640 1 -- 172.21.15.97:0/2018554734 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4031a00 con 0x7f12e40cffe0 2024-03-29T04:08:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.216+0000 7f12ec5af640 1 -- 172.21.15.97:0/2018554734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12e40cffe0 msgr2=0x7f12e409b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.216+0000 7f12ec5af640 1 --2- 172.21.15.97:0/2018554734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12e40cffe0 0x7f12e409b510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f12e40d0770 tx=0x7f12d402b880 comp rx=0 tx=0).stop 2024-03-29T04:08:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.216+0000 7f12ec5af640 1 -- 172.21.15.97:0/2018554734 shutdown_connections 2024-03-29T04:08:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.216+0000 7f12ec5af640 1 --2- 172.21.15.97:0/2018554734 >> v2:172.21.15.154:3300/0 conn(0x7f12e409ba50 0x7f12e40cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.216+0000 7f12ec5af640 1 --2- 172.21.15.97:0/2018554734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12e40cffe0 0x7f12e409b510 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.216+0000 7f12ec5af640 1 -- 172.21.15.97:0/2018554734 >> 172.21.15.97:0/2018554734 conn(0x7f12e40b3f00 msgr2=0x7f12e40b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:28.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.216+0000 7f12ec5af640 1 -- 172.21.15.97:0/2018554734 shutdown_connections 2024-03-29T04:08:28.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.216+0000 7f12ec5af640 1 -- 172.21.15.97:0/2018554734 wait complete. 2024-03-29T04:08:28.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.217+0000 7f12ec5af640 1 Processor -- start 2024-03-29T04:08:28.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.217+0000 7f12ec5af640 1 -- start start 2024-03-29T04:08:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12ec5af640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12e409ba50 0x7f12e4156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:28.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12ec5af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12e40cffe0 0x7f12e415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12ec5af640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12e415dc80 0x7f12e40cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12ec5af640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f12e409aff0 con 0x7f12e40cffe0 2024-03-29T04:08:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12ec5af640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f12e409ad10 con 0x7f12e409ba50 2024-03-29T04:08:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12ec5af640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f12e4070130 con 0x7f12e415dc80 2024-03-29T04:08:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12e9b4a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12e40cffe0 0x7f12e415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12e9b4a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12e40cffe0 0x7f12e415cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50238/0 (socket says 172.21.15.97:50238) 2024-03-29T04:08:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12e9b4a640 1 -- 172.21.15.97:0/1449282817 learned_addr learned my addr 172.21.15.97:0/1449282817 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12ea34b640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12e409ba50 0x7f12e4156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12eab4c640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12e415dc80 0x7f12e40cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12e9b4a640 1 -- 172.21.15.97:0/1449282817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12e415dc80 msgr2=0x7f12e40cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:28.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12e9b4a640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12e415dc80 0x7f12e40cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12e9b4a640 1 -- 172.21.15.97:0/1449282817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12e409ba50 msgr2=0x7f12e4156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12e9b4a640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12e409ba50 0x7f12e4156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.218+0000 7f12e9b4a640 1 -- 172.21.15.97:0/1449282817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e4159230 con 0x7f12e40cffe0 2024-03-29T04:08:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.219+0000 7f12e9b4a640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12e40cffe0 0x7f12e415cf90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f12d4000c00 tx=0x7f12d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.219+0000 7f12daffd640 1 -- 172.21.15.97:0/1449282817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4032880 con 0x7f12e40cffe0 2024-03-29T04:08:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.219+0000 7f12daffd640 1 -- 172.21.15.97:0/1449282817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d4032a20 con 0x7f12e40cffe0 2024-03-29T04:08:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.219+0000 7f12daffd640 1 -- 172.21.15.97:0/1449282817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4032e10 con 0x7f12e40cffe0 2024-03-29T04:08:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.219+0000 7f12ec5af640 1 -- 172.21.15.97:0/1449282817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e4159510 con 0x7f12e40cffe0 2024-03-29T04:08:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.219+0000 7f12ec5af640 1 -- 172.21.15.97:0/1449282817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e40cc230 con 0x7f12e40cffe0 2024-03-29T04:08:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.220+0000 7f12ec5af640 1 -- 172.21.15.97:0/1449282817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e4068640 con 0x7f12e40cffe0 2024-03-29T04:08:28.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.220+0000 7f12daffd640 1 -- 172.21.15.97:0/1449282817 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f12d40453f0 con 0x7f12e40cffe0 2024-03-29T04:08:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.220+0000 7f12daffd640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f12c806c530 0x7f12c806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.221+0000 7f12daffd640 1 -- 172.21.15.97:0/1449282817 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f12d40ad2d0 con 0x7f12e40cffe0 2024-03-29T04:08:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.221+0000 7f12ea34b640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f12c806c530 0x7f12c806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:28.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.221+0000 7f12ea34b640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f12c806c530 0x7f12c806e9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f12e0009f20 tx=0x7f12e00094d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:28.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.225+0000 7f12daffd640 1 -- 172.21.15.97:0/1449282817 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f12d4040070 con 0x7f12e40cffe0 2024-03-29T04:08:28.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.399+0000 7f12ec5af640 1 -- 172.21.15.97:0/1449282817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f12e40b4ab0 con 0x7f12e40cffe0 2024-03-29T04:08:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.595+0000 7f12daffd640 1 -- 172.21.15.97:0/1449282817 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) ==== 138+0+0 (secure 0 0 0) 0x7f12d407bcd0 con 0x7f12e40cffe0 2024-03-29T04:08:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-03-29T04:08:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.597+0000 7f12ec5af640 1 -- 172.21.15.97:0/1449282817 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f12c806c530 msgr2=0x7f12c806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.597+0000 7f12ec5af640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f12c806c530 0x7f12c806e9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f12e0009f20 tx=0x7f12e00094d0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.597+0000 7f12ec5af640 1 -- 172.21.15.97:0/1449282817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12e40cffe0 msgr2=0x7f12e415cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.597+0000 7f12ec5af640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12e40cffe0 0x7f12e415cf90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f12d4000c00 tx=0x7f12d40046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.597+0000 7f12ec5af640 1 -- 172.21.15.97:0/1449282817 shutdown_connections 2024-03-29T04:08:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.597+0000 7f12ec5af640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f12c806c530 0x7f12c806e9f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.597+0000 7f12ec5af640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12e415dc80 0x7f12e40cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.597+0000 7f12ec5af640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12e40cffe0 0x7f12e415cf90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.597+0000 7f12ec5af640 1 --2- 172.21.15.97:0/1449282817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12e409ba50 0x7f12e4156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.597+0000 7f12ec5af640 1 -- 172.21.15.97:0/1449282817 >> 172.21.15.97:0/1449282817 conn(0x7f12e40b3f00 msgr2=0x7f12e40cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.597+0000 7f12ec5af640 1 -- 172.21.15.97:0/1449282817 shutdown_connections 2024-03-29T04:08:28.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.597+0000 7f12ec5af640 1 -- 172.21.15.97:0/1449282817 wait complete. 2024-03-29T04:08:28.590 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T04:08:28.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.717+0000 7f3183a88640 1 Processor -- start 2024-03-29T04:08:28.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.717+0000 7f3183a88640 1 -- start start 2024-03-29T04:08:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3183a88640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f317c09aa30 0x7f317c09aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3183a88640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f317c0d0bd0 0x7f317c0d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3183a88640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f317c0d06f0 con 0x7f317c161f50 2024-03-29T04:08:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3183a88640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f317c070250 con 0x7f317c0d0bd0 2024-03-29T04:08:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3183a88640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f317c15f300 con 0x7f317c09aa30 2024-03-29T04:08:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3182025640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f317c0d0bd0 0x7f317c0d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3181824640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f317c161f50 0x7f317c06fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59162/0 (socket says 172.21.15.97:59162) 2024-03-29T04:08:28.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3182025640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f317c0d0bd0 0x7f317c0d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50252/0 (socket says 172.21.15.97:50252) 2024-03-29T04:08:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3181824640 1 -- 172.21.15.97:0/2418711265 learned_addr learned my addr 172.21.15.97:0/2418711265 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3182025640 1 -- 172.21.15.97:0/2418711265 >> v2:172.21.15.154:3300/0 conn(0x7f317c09aa30 msgr2=0x7f317c09aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3182025640 1 --2- 172.21.15.97:0/2418711265 >> v2:172.21.15.154:3300/0 conn(0x7f317c09aa30 0x7f317c09aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3182025640 1 -- 172.21.15.97:0/2418711265 >> v1:172.21.15.97:6789/0 conn(0x7f317c161f50 legacy=0x7f317c06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3180822640 1 -- 172.21.15.97:0/2418711265 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1372902834 0 0) 0x7f317c0d06f0 con 0x7f317c161f50 2024-03-29T04:08:28.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.718+0000 7f3182025640 1 -- 172.21.15.97:0/2418711265 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c15f5e0 con 0x7f317c0d0bd0 2024-03-29T04:08:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.719+0000 7f3182025640 1 --2- 172.21.15.97:0/2418711265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f317c0d0bd0 0x7f317c0d0fe0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f317c073a00 tx=0x7f317402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939de929f9f8471a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:28.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.719+0000 7f3180822640 1 -- 172.21.15.97:0/2418711265 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174039070 con 0x7f317c0d0bd0 2024-03-29T04:08:28.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.719+0000 7f3180822640 1 -- 172.21.15.97:0/2418711265 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f317402cbf0 con 0x7f317c0d0bd0 2024-03-29T04:08:28.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.719+0000 7f3180822640 1 -- 172.21.15.97:0/2418711265 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174034440 con 0x7f317c0d0bd0 2024-03-29T04:08:28.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.719+0000 7f3183a88640 1 -- 172.21.15.97:0/2418711265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f317c0d0bd0 msgr2=0x7f317c0d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:28.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.719+0000 7f3183a88640 1 --2- 172.21.15.97:0/2418711265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f317c0d0bd0 0x7f317c0d0fe0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f317c073a00 tx=0x7f317402be40 comp rx=0 tx=0).stop 2024-03-29T04:08:28.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.720+0000 7f3183a88640 1 -- 172.21.15.97:0/2418711265 shutdown_connections 2024-03-29T04:08:28.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.720+0000 7f3183a88640 1 --2- 172.21.15.97:0/2418711265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f317c0d0bd0 0x7f317c0d0fe0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.720+0000 7f3183a88640 1 --2- 172.21.15.97:0/2418711265 >> v2:172.21.15.154:3300/0 conn(0x7f317c09aa30 0x7f317c09aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.720+0000 7f3183a88640 1 -- 172.21.15.97:0/2418711265 >> 172.21.15.97:0/2418711265 conn(0x7f317c0b3fa0 msgr2=0x7f317c0b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:28.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.720+0000 7f3183a88640 1 -- 172.21.15.97:0/2418711265 shutdown_connections 2024-03-29T04:08:28.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.720+0000 7f3183a88640 1 -- 172.21.15.97:0/2418711265 wait complete. 2024-03-29T04:08:28.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.721+0000 7f3183a88640 1 Processor -- start 2024-03-29T04:08:28.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.721+0000 7f3183a88640 1 -- start start 2024-03-29T04:08:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.721+0000 7f3183a88640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f317c09aa30 0x7f317c0b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.721+0000 7f3183a88640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c0d0bd0 0x7f317c0b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.721+0000 7f3183a88640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f317c161f50 0x7f317c0c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.721+0000 7f3183a88640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f317c160220 con 0x7f317c09aa30 2024-03-29T04:08:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.721+0000 7f3183a88640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f317c15fba0 con 0x7f317c161f50 2024-03-29T04:08:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.721+0000 7f3183a88640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f317c1603a0 con 0x7f317c0d0bd0 2024-03-29T04:08:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f3181824640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f317c09aa30 0x7f317c0b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f3181824640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f317c09aa30 0x7f317c0b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50268/0 (socket says 172.21.15.97:50268) 2024-03-29T04:08:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f3181824640 1 -- 172.21.15.97:0/377679563 learned_addr learned my addr 172.21.15.97:0/377679563 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f3182025640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f317c161f50 0x7f317c0c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f3181824640 1 -- 172.21.15.97:0/377679563 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c0d0bd0 msgr2=0x7f317c0b7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:28.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f3181824640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c0d0bd0 0x7f317c0b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f3181824640 1 -- 172.21.15.97:0/377679563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f317c161f50 msgr2=0x7f317c0c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f3181824640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f317c161f50 0x7f317c0c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f3181824640 1 -- 172.21.15.97:0/377679563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c0b8100 con 0x7f317c09aa30 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f3182025640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f317c161f50 0x7f317c0c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f3181824640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f317c09aa30 0x7f317c0b0920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f317800c430 tx=0x7f317800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f31727fc640 1 -- 172.21.15.97:0/377679563 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31780074e0 con 0x7f317c09aa30 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f31727fc640 1 -- 172.21.15.97:0/377679563 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31780047e0 con 0x7f317c09aa30 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f3183a88640 1 -- 172.21.15.97:0/377679563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317c0b6b20 con 0x7f317c09aa30 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f31727fc640 1 -- 172.21.15.97:0/377679563 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3178004bd0 con 0x7f317c09aa30 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.722+0000 7f3183a88640 1 -- 172.21.15.97:0/377679563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317c0b70e0 con 0x7f317c09aa30 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.723+0000 7f3183a88640 1 -- 172.21.15.97:0/377679563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317c068de0 con 0x7f317c09aa30 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.724+0000 7f31727fc640 1 -- 172.21.15.97:0/377679563 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3178011070 con 0x7f317c09aa30 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.724+0000 7f31727fc640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f316006c560 0x7f316006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:28.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.724+0000 7f31727fc640 1 -- 172.21.15.97:0/377679563 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5227+0+0 (secure 0 0 0) 0x7f317808b030 con 0x7f317c09aa30 2024-03-29T04:08:28.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.724+0000 7f3181023640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f316006c560 0x7f316006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:28.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.725+0000 7f3181023640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f316006c560 0x7f316006ea20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f317c0a0590 tx=0x7f316c006bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:28.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.729+0000 7f31727fc640 1 -- 172.21.15.97:0/377679563 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f317c068de0 con 0x7f317c09aa30 2024-03-29T04:08:28.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:28.905+0000 7f3183a88640 1 -- 172.21.15.97:0/377679563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f317c0a05c0 con 0x7f317c09aa30 2024-03-29T04:08:29.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.600+0000 7f31727fc640 1 -- 172.21.15.97:0/377679563 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) ==== 170+0+0 (secure 0 0 0) 0x7f31780598c0 con 0x7f317c09aa30 2024-03-29T04:08:29.580 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-03-29T04:08:29.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.603+0000 7f3183a88640 1 -- 172.21.15.97:0/377679563 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f316006c560 msgr2=0x7f316006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:29.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.603+0000 7f3183a88640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f316006c560 0x7f316006ea20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f317c0a0590 tx=0x7f316c006bf0 comp rx=0 tx=0).stop 2024-03-29T04:08:29.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.603+0000 7f3183a88640 1 -- 172.21.15.97:0/377679563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f317c09aa30 msgr2=0x7f317c0b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:29.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.603+0000 7f3183a88640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f317c09aa30 0x7f317c0b0920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f317800c430 tx=0x7f317800c8f0 comp rx=0 tx=0).stop 2024-03-29T04:08:29.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.603+0000 7f3183a88640 1 -- 172.21.15.97:0/377679563 shutdown_connections 2024-03-29T04:08:29.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.603+0000 7f3183a88640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f316006c560 0x7f316006ea20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:29.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.603+0000 7f3183a88640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f317c161f50 0x7f317c0c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:29.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.603+0000 7f3183a88640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f317c0d0bd0 0x7f317c0b7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:29.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.603+0000 7f3183a88640 1 --2- 172.21.15.97:0/377679563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f317c09aa30 0x7f317c0b0920 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:29.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.603+0000 7f3183a88640 1 -- 172.21.15.97:0/377679563 >> 172.21.15.97:0/377679563 conn(0x7f317c0b3fa0 msgr2=0x7f317c15e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:29.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.603+0000 7f3183a88640 1 -- 172.21.15.97:0/377679563 shutdown_connections 2024-03-29T04:08:29.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.603+0000 7f3183a88640 1 -- 172.21.15.97:0/377679563 wait complete. 2024-03-29T04:08:29.604 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T04:08:29.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.729+0000 7fb3bf5c8640 1 Processor -- start 2024-03-29T04:08:29.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.730+0000 7fb3bf5c8640 1 -- start start 2024-03-29T04:08:29.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.730+0000 7fb3bf5c8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3b80cfde0 0x7fb3b80d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:29.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.730+0000 7fb3bf5c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b80d0f90 0x7fb3b80d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:29.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.730+0000 7fb3bf5c8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b815ed00 con 0x7fb3b815e920 2024-03-29T04:08:29.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.730+0000 7fb3bf5c8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b80d21e0 con 0x7fb3b80cfde0 2024-03-29T04:08:29.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.730+0000 7fb3bf5c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b80d2480 con 0x7fb3b80d0f90 2024-03-29T04:08:29.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.731+0000 7fb3bcb63640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3b80cfde0 0x7fb3b80d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:29.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.731+0000 7fb3bcb63640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3b80cfde0 0x7fb3b80d0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50284/0 (socket says 172.21.15.97:50284) 2024-03-29T04:08:29.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.731+0000 7fb3bcb63640 1 -- 172.21.15.97:0/2019841430 learned_addr learned my addr 172.21.15.97:0/2019841430 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:29.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.731+0000 7fb3bdb65640 1 --2- 172.21.15.97:0/2019841430 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b80d0f90 0x7fb3b80d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:29.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.731+0000 7fb3bcb63640 1 -- 172.21.15.97:0/2019841430 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b80d0f90 msgr2=0x7fb3b80d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:29.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.731+0000 7fb3bcb63640 1 --2- 172.21.15.97:0/2019841430 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b80d0f90 0x7fb3b80d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:29.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.731+0000 7fb3bcb63640 1 -- 172.21.15.97:0/2019841430 >> v1:172.21.15.154:6789/0 conn(0x7fb3b815e920 legacy=0x7fb3b80d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:29.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.731+0000 7fb3bcb63640 1 -- 172.21.15.97:0/2019841430 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b80d2760 con 0x7fb3b80cfde0 2024-03-29T04:08:29.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.731+0000 7fb3bdb65640 1 --2- 172.21.15.97:0/2019841430 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b80d0f90 0x7fb3b80d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:29.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.732+0000 7fb3bcb63640 1 --2- 172.21.15.97:0/2019841430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b80cfde0 0x7fb3b80d0240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009870 tx=0x7fb3a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16654838cd3c641a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:29.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.732+0000 7fb3affff640 1 -- 172.21.15.97:0/2019841430 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8034070 con 0x7fb3b80cfde0 2024-03-29T04:08:29.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.732+0000 7fb3affff640 1 -- 172.21.15.97:0/2019841430 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a8030840 con 0x7fb3b80cfde0 2024-03-29T04:08:29.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.732+0000 7fb3affff640 1 -- 172.21.15.97:0/2019841430 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8030c10 con 0x7fb3b80cfde0 2024-03-29T04:08:29.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.732+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/2019841430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b80cfde0 msgr2=0x7fb3b80d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:29.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.732+0000 7fb3bf5c8640 1 --2- 172.21.15.97:0/2019841430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b80cfde0 0x7fb3b80d0240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009870 tx=0x7fb3a802b980 comp rx=0 tx=0).stop 2024-03-29T04:08:29.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.732+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/2019841430 shutdown_connections 2024-03-29T04:08:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.732+0000 7fb3bf5c8640 1 --2- 172.21.15.97:0/2019841430 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b80d0f90 0x7fb3b80d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.732+0000 7fb3bf5c8640 1 --2- 172.21.15.97:0/2019841430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b80cfde0 0x7fb3b80d0240 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.732+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/2019841430 >> 172.21.15.97:0/2019841430 conn(0x7fb3b808bfc0 msgr2=0x7fb3b80d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.732+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/2019841430 shutdown_connections 2024-03-29T04:08:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.732+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/2019841430 wait complete. 2024-03-29T04:08:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.733+0000 7fb3bf5c8640 1 Processor -- start 2024-03-29T04:08:29.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.734+0000 7fb3bf5c8640 1 -- start start 2024-03-29T04:08:29.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.734+0000 7fb3bf5c8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b80cfde0 0x7fb3b80b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:29.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.734+0000 7fb3bf5c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b80d0f90 0x7fb3b8155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:29.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.734+0000 7fb3bf5c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b815e920 0x7fb3b80b9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:29.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.734+0000 7fb3bf5c8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b80d4ef0 con 0x7fb3b80cfde0 2024-03-29T04:08:29.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.734+0000 7fb3bf5c8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b80d2d20 con 0x7fb3b80d0f90 2024-03-29T04:08:29.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.734+0000 7fb3bf5c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b80d5070 con 0x7fb3b815e920 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.734+0000 7fb3bd364640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b80cfde0 0x7fb3b80b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.734+0000 7fb3bd364640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b80cfde0 0x7fb3b80b6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50290/0 (socket says 172.21.15.97:50290) 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.734+0000 7fb3bd364640 1 -- 172.21.15.97:0/448371822 learned_addr learned my addr 172.21.15.97:0/448371822 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3bcb63640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b80d0f90 0x7fb3b8155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3bdb65640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b815e920 0x7fb3b80b9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3bd364640 1 -- 172.21.15.97:0/448371822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b815e920 msgr2=0x7fb3b80b9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3bd364640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b815e920 0x7fb3b80b9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3bd364640 1 -- 172.21.15.97:0/448371822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b80d0f90 msgr2=0x7fb3b8155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3bd364640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b80d0f90 0x7fb3b8155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3bd364640 1 -- 172.21.15.97:0/448371822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b80b51b0 con 0x7fb3b80cfde0 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3bcb63640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b80d0f90 0x7fb3b8155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3bd364640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b80cfde0 0x7fb3b80b6210 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400c790 tx=0x7fb3b400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3adffb640 1 -- 172.21.15.97:0/448371822 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4016020 con 0x7fb3b80cfde0 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3adffb640 1 -- 172.21.15.97:0/448371822 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b4006d50 con 0x7fb3b80cfde0 2024-03-29T04:08:29.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/448371822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b80b5450 con 0x7fb3b80cfde0 2024-03-29T04:08:29.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3adffb640 1 -- 172.21.15.97:0/448371822 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4003bb0 con 0x7fb3b80cfde0 2024-03-29T04:08:29.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.735+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/448371822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b80b5a10 con 0x7fb3b80cfde0 2024-03-29T04:08:29.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.736+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/448371822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b80a2560 con 0x7fb3b80cfde0 2024-03-29T04:08:29.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.737+0000 7fb3adffb640 1 -- 172.21.15.97:0/448371822 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb3b400a850 con 0x7fb3b80cfde0 2024-03-29T04:08:29.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.737+0000 7fb3adffb640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb39c06c530 0x7fb39c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:29.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.737+0000 7fb3adffb640 1 -- 172.21.15.97:0/448371822 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5609+0+0 (secure 0 0 0) 0x7fb3b408b320 con 0x7fb3b80cfde0 2024-03-29T04:08:29.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.737+0000 7fb3bcb63640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb39c06c530 0x7fb39c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:29.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.738+0000 7fb3bcb63640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb39c06c530 0x7fb39c06e9f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8036000 tx=0x7fb3a803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:29.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.742+0000 7fb3adffb640 1 -- 172.21.15.97:0/448371822 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb3b4020020 con 0x7fb3b80cfde0 2024-03-29T04:08:29.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:29.954+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/448371822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb3b80c7ee0 con 0x7fb3b80cfde0 2024-03-29T04:08:30.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.601+0000 7fb3adffb640 1 -- 172.21.15.97:0/448371822 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) ==== 172+0+0 (secure 0 0 0) 0x7fb3b4019310 con 0x7fb3b80cfde0 2024-03-29T04:08:30.582 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 4 and data pool 5 2024-03-29T04:08:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.605+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/448371822 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb39c06c530 msgr2=0x7fb39c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.605+0000 7fb3bf5c8640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb39c06c530 0x7fb39c06e9f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8036000 tx=0x7fb3a803b040 comp rx=0 tx=0).stop 2024-03-29T04:08:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.605+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/448371822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b80cfde0 msgr2=0x7fb3b80b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.605+0000 7fb3bf5c8640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b80cfde0 0x7fb3b80b6210 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400c790 tx=0x7fb3b400cc50 comp rx=0 tx=0).stop 2024-03-29T04:08:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.605+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/448371822 shutdown_connections 2024-03-29T04:08:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.605+0000 7fb3bf5c8640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb39c06c530 0x7fb39c06e9f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.605+0000 7fb3bf5c8640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3b815e920 0x7fb3b80b9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.605+0000 7fb3bf5c8640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3b80d0f90 0x7fb3b8155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.605+0000 7fb3bf5c8640 1 --2- 172.21.15.97:0/448371822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b80cfde0 0x7fb3b80b6210 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.605+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/448371822 >> 172.21.15.97:0/448371822 conn(0x7fb3b808bfc0 msgr2=0x7fb3b80d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.606+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/448371822 shutdown_connections 2024-03-29T04:08:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.606+0000 7fb3bf5c8640 1 -- 172.21.15.97:0/448371822 wait complete. 2024-03-29T04:08:30.616 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:30.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.740+0000 7f7a3384a640 1 Processor -- start 2024-03-29T04:08:30.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.740+0000 7f7a3384a640 1 -- start start 2024-03-29T04:08:30.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a3384a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c15f9d0 0x7f7a2c162d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:30.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a3384a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7a2c0d4e20 0x7f7a2c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:30.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a3384a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a2c0cf390 con 0x7f7a2c0d1150 2024-03-29T04:08:30.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a3384a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f7a2c0d06d0 con 0x7f7a2c0d4e20 2024-03-29T04:08:30.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a3384a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7a2c0d0240 con 0x7f7a2c15f9d0 2024-03-29T04:08:30.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a315e6640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7a2c0d1150 0x7f7a2c0d2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55758/0 (socket says 172.21.15.97:55758) 2024-03-29T04:08:30.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a315e6640 1 -- 172.21.15.97:0/2374153707 learned_addr learned my addr 172.21.15.97:0/2374153707 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:30.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3766849251 0 0) 0x7f7a2c0cf390 con 0x7f7a2c0d1150 2024-03-29T04:08:30.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a18003520 con 0x7f7a2c0d1150 2024-03-29T04:08:30.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a31de7640 1 --2- 172.21.15.97:0/2374153707 >> v2:172.21.15.154:3300/0 conn(0x7f7a2c0d4e20 0x7f7a2c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a30de5640 1 --2- 172.21.15.97:0/2374153707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c15f9d0 0x7f7a2c162d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3767484435 0 0) 0x7f7a18003520 con 0x7f7a2c0d1150 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c15f9d0 msgr2=0x7f7a2c162d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 --2- 172.21.15.97:0/2374153707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c15f9d0 0x7f7a2c162d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 >> v2:172.21.15.154:3300/0 conn(0x7f7a2c0d4e20 msgr2=0x7f7a2c0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 --2- 172.21.15.97:0/2374153707 >> v2:172.21.15.154:3300/0 conn(0x7f7a2c0d4e20 0x7f7a2c0d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c0cf630 con 0x7f7a2c0d1150 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f7a28001670 con 0x7f7a2c0d1150 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a31de7640 1 --2- 172.21.15.97:0/2374153707 >> v2:172.21.15.154:3300/0 conn(0x7f7a2c0d4e20 0x7f7a2c0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 >> v1:172.21.15.97:6789/0 conn(0x7f7a2c0d1150 legacy=0x7f7a2c0d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 --2- 172.21.15.97:0/2374153707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c15f9d0 0x7f7a180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 --2- 172.21.15.97:0/2374153707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a18003fe0 0x7f7a18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 --2- 172.21.15.97:0/2374153707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a18004d40 0x7f7a180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c0d06d0 con 0x7f7a18004d40 2024-03-29T04:08:30.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c0d0240 con 0x7f7a18003fe0 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.741+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1800bac0 con 0x7f7a2c15f9d0 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.742+0000 7f7a31de7640 1 --2- 172.21.15.97:0/2374153707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a18004d40 0x7f7a180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.742+0000 7f7a315e6640 1 --2- 172.21.15.97:0/2374153707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c15f9d0 0x7f7a180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.742+0000 7f7a30de5640 1 --2- 172.21.15.97:0/2374153707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a18003fe0 0x7f7a18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.742+0000 7f7a31de7640 1 -- 172.21.15.97:0/2374153707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c15f9d0 msgr2=0x7f7a180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.742+0000 7f7a31de7640 1 --2- 172.21.15.97:0/2374153707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c15f9d0 0x7f7a180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.742+0000 7f7a31de7640 1 -- 172.21.15.97:0/2374153707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a18003fe0 msgr2=0x7f7a18004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.742+0000 7f7a31de7640 1 --2- 172.21.15.97:0/2374153707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a18003fe0 0x7f7a18004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.742+0000 7f7a31de7640 1 -- 172.21.15.97:0/2374153707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a18003520 con 0x7f7a18004d40 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.743+0000 7f7a31de7640 1 --2- 172.21.15.97:0/2374153707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a18004d40 0x7f7a180051c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7a240073b0 tx=0x7f7a18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fda31ad1cbc337 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.743+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24039070 con 0x7f7a18004d40 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.743+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a2402cd90 con 0x7f7a18004d40 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.743+0000 7f7a23fff640 1 -- 172.21.15.97:0/2374153707 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24009c80 con 0x7f7a18004d40 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.743+0000 7f7a3384a640 1 -- 172.21.15.97:0/2374153707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a18004d40 msgr2=0x7f7a180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.743+0000 7f7a3384a640 1 --2- 172.21.15.97:0/2374153707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a18004d40 0x7f7a180051c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7a240073b0 tx=0x7f7a18005830 comp rx=0 tx=0).stop 2024-03-29T04:08:30.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.743+0000 7f7a31de7640 1 -- 172.21.15.97:0/2374153707 reap_dead start 2024-03-29T04:08:30.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.743+0000 7f7a3384a640 1 -- 172.21.15.97:0/2374153707 shutdown_connections 2024-03-29T04:08:30.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.743+0000 7f7a3384a640 1 -- 172.21.15.97:0/2374153707 >> 172.21.15.97:0/2374153707 conn(0x7f7a2c08bfc0 msgr2=0x7f7a2c0d4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:30.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.743+0000 7f7a3384a640 1 -- 172.21.15.97:0/2374153707 shutdown_connections 2024-03-29T04:08:30.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.743+0000 7f7a3384a640 1 -- 172.21.15.97:0/2374153707 wait complete. 2024-03-29T04:08:30.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.745+0000 7f7a3384a640 1 Processor -- start 2024-03-29T04:08:30.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.745+0000 7f7a3384a640 1 -- start start 2024-03-29T04:08:30.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.745+0000 7f7a3384a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c156fe0 0x7f7a2c0b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:30.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.745+0000 7f7a3384a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a2c0b4080 0x7f7a2c0b6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.745+0000 7f7a3384a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c161550 0x7f7a2c161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.745+0000 7f7a3384a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c0cfed0 con 0x7f7a2c0b4080 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.745+0000 7f7a3384a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c0cfbf0 con 0x7f7a2c161550 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.745+0000 7f7a3384a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f7a2c0d1530 con 0x7f7a2c156fe0 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.745+0000 7f7a30de5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a2c0b4080 0x7f7a2c0b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.745+0000 7f7a30de5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a2c0b4080 0x7f7a2c0b6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51164/0 (socket says 172.21.15.97:51164) 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.745+0000 7f7a30de5640 1 -- 172.21.15.97:0/2592134902 learned_addr learned my addr 172.21.15.97:0/2592134902 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.746+0000 7f7a30de5640 1 -- 172.21.15.97:0/2592134902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c156fe0 msgr2=0x7f7a2c0b75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.746+0000 7f7a30de5640 1 --2- 172.21.15.97:0/2592134902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c156fe0 0x7f7a2c0b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.746+0000 7f7a30de5640 1 -- 172.21.15.97:0/2592134902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c161550 msgr2=0x7f7a2c161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.746+0000 7f7a31de7640 1 --2- 172.21.15.97:0/2592134902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c161550 0x7f7a2c161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.746+0000 7f7a30de5640 1 --2- 172.21.15.97:0/2592134902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c161550 0x7f7a2c161a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.746+0000 7f7a30de5640 1 -- 172.21.15.97:0/2592134902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c0b4560 con 0x7f7a2c0b4080 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.746+0000 7f7a30de5640 1 --2- 172.21.15.97:0/2592134902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a2c0b4080 0x7f7a2c0b6840 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00b7f0 tx=0x7f7a1c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:30.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.746+0000 7f7a21ffb640 1 -- 172.21.15.97:0/2592134902 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c018070 con 0x7f7a2c0b4080 2024-03-29T04:08:30.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.746+0000 7f7a21ffb640 1 -- 172.21.15.97:0/2592134902 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a1c010830 con 0x7f7a2c0b4080 2024-03-29T04:08:30.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.746+0000 7f7a3384a640 1 -- 172.21.15.97:0/2592134902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c0b4840 con 0x7f7a2c0b4080 2024-03-29T04:08:30.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.746+0000 7f7a21ffb640 1 -- 172.21.15.97:0/2592134902 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c010c40 con 0x7f7a2c0b4080 2024-03-29T04:08:30.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.746+0000 7f7a3384a640 1 -- 172.21.15.97:0/2592134902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c0b4e00 con 0x7f7a2c0b4080 2024-03-29T04:08:30.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.747+0000 7f7a3384a640 1 -- 172.21.15.97:0/2592134902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c068640 con 0x7f7a2c0b4080 2024-03-29T04:08:30.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.747+0000 7f7a21ffb640 1 -- 172.21.15.97:0/2592134902 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7a1c01b070 con 0x7f7a2c0b4080 2024-03-29T04:08:30.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.748+0000 7f7a21ffb640 1 --2- 172.21.15.97:0/2592134902 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7a1006c530 0x7f7a1006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:30.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.748+0000 7f7a21ffb640 1 -- 172.21.15.97:0/2592134902 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f7a1c08bdb0 con 0x7f7a2c0b4080 2024-03-29T04:08:30.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.748+0000 7f7a315e6640 1 --2- 172.21.15.97:0/2592134902 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7a1006c530 0x7f7a1006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:30.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.748+0000 7f7a315e6640 1 --2- 172.21.15.97:0/2592134902 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7a1006c530 0x7f7a1006e9f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7a28000f60 tx=0x7f7a28000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:30.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.752+0000 7f7a21ffb640 1 -- 172.21.15.97:0/2592134902 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7a1c05a460 con 0x7f7a2c0b4080 2024-03-29T04:08:30.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.929+0000 7f7a3384a640 1 -- 172.21.15.97:0/2592134902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a2c0b6340 con 0x7f7a2c0b4080 2024-03-29T04:08:30.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.930+0000 7f7a21ffb640 1 -- 172.21.15.97:0/2592134902 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16682 (secure 0 0 0) 0x7f7a1c059d60 con 0x7f7a2c0b4080 2024-03-29T04:08:30.910 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:30.910 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":31,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:08:29.955551+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:08:28.403360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T04:08:28.909709+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.932+0000 7f7a3384a640 1 -- 172.21.15.97:0/2592134902 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7a1006c530 msgr2=0x7f7a1006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.932+0000 7f7a3384a640 1 --2- 172.21.15.97:0/2592134902 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7a1006c530 0x7f7a1006e9f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7a28000f60 tx=0x7f7a28000ed0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.932+0000 7f7a3384a640 1 -- 172.21.15.97:0/2592134902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a2c0b4080 msgr2=0x7f7a2c0b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.932+0000 7f7a3384a640 1 --2- 172.21.15.97:0/2592134902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a2c0b4080 0x7f7a2c0b6840 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00b7f0 tx=0x7f7a1c00bcb0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.932+0000 7f7a3384a640 1 -- 172.21.15.97:0/2592134902 shutdown_connections 2024-03-29T04:08:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.932+0000 7f7a3384a640 1 --2- 172.21.15.97:0/2592134902 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f7a1006c530 0x7f7a1006e9f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.932+0000 7f7a3384a640 1 --2- 172.21.15.97:0/2592134902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a2c161550 0x7f7a2c161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.932+0000 7f7a3384a640 1 --2- 172.21.15.97:0/2592134902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a2c0b4080 0x7f7a2c0b6840 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.932+0000 7f7a3384a640 1 --2- 172.21.15.97:0/2592134902 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a2c156fe0 0x7f7a2c0b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.932+0000 7f7a3384a640 1 -- 172.21.15.97:0/2592134902 >> 172.21.15.97:0/2592134902 conn(0x7f7a2c08bfc0 msgr2=0x7f7a2c0d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.932+0000 7f7a3384a640 1 -- 172.21.15.97:0/2592134902 shutdown_connections 2024-03-29T04:08:30.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:30.932+0000 7f7a3384a640 1 -- 172.21.15.97:0/2592134902 wait complete. 2024-03-29T04:08:30.923 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:31.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.044+0000 7fb299a74640 1 Processor -- start 2024-03-29T04:08:31.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.044+0000 7fb299a74640 1 -- start start 2024-03-29T04:08:31.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb299a74640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb29406f1e0 0x7fb2940b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:31.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb299a74640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb29406e1c0 0x7fb29406e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:31.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb299a74640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2940b3c10 con 0x7fb294144790 2024-03-29T04:08:31.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb299a74640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fb2940b3ef0 con 0x7fb29406f1e0 2024-03-29T04:08:31.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb299a74640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb2940b41d0 con 0x7fb29406e1c0 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb292ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb29406f1e0 0x7fb2940b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb292ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb29406f1e0 0x7fb2940b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51166/0 (socket says 172.21.15.97:51166) 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb292ffd640 1 -- 172.21.15.97:0/3494967658 learned_addr learned my addr 172.21.15.97:0/3494967658 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb2937fe640 1 --1- 172.21.15.97:0/3494967658 >> v1:172.21.15.97:6789/0 conn(0x7fb294144790 0x7fb2940b4cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55766/0 (socket says 172.21.15.97:55766) 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb293fff640 1 --2- 172.21.15.97:0/3494967658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb29406e1c0 0x7fb29406e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb292ffd640 1 -- 172.21.15.97:0/3494967658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb29406e1c0 msgr2=0x7fb29406e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb292ffd640 1 --2- 172.21.15.97:0/3494967658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb29406e1c0 0x7fb29406e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb292ffd640 1 -- 172.21.15.97:0/3494967658 >> v1:172.21.15.97:6789/0 conn(0x7fb294144790 legacy=0x7fb2940b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb292ffd640 1 -- 172.21.15.97:0/3494967658 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2940b44b0 con 0x7fb29406f1e0 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.045+0000 7fb2927fc640 1 -- 172.21.15.97:0/3494967658 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 228232062 0 0) 0x7fb2940b3c10 con 0x7fb294144790 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.046+0000 7fb292ffd640 1 --2- 172.21.15.97:0/3494967658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29406f1e0 0x7fb2940b32c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb28800bb00 tx=0x7fb28802df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e43e87e7f9c5df3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.046+0000 7fb2927fc640 1 -- 172.21.15.97:0/3494967658 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288034070 con 0x7fb29406f1e0 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.046+0000 7fb2927fc640 1 -- 172.21.15.97:0/3494967658 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28802ece0 con 0x7fb29406f1e0 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.046+0000 7fb2927fc640 1 -- 172.21.15.97:0/3494967658 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288038620 con 0x7fb29406f1e0 2024-03-29T04:08:31.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.046+0000 7fb299a74640 1 -- 172.21.15.97:0/3494967658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29406f1e0 msgr2=0x7fb2940b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.046+0000 7fb299a74640 1 --2- 172.21.15.97:0/3494967658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29406f1e0 0x7fb2940b32c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb28800bb00 tx=0x7fb28802df30 comp rx=0 tx=0).stop 2024-03-29T04:08:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.047+0000 7fb299a74640 1 -- 172.21.15.97:0/3494967658 shutdown_connections 2024-03-29T04:08:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.047+0000 7fb299a74640 1 --2- 172.21.15.97:0/3494967658 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb29406e1c0 0x7fb29406e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.047+0000 7fb299a74640 1 --2- 172.21.15.97:0/3494967658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29406f1e0 0x7fb2940b32c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.047+0000 7fb299a74640 1 -- 172.21.15.97:0/3494967658 >> 172.21.15.97:0/3494967658 conn(0x7fb29406b510 msgr2=0x7fb29406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.047+0000 7fb299a74640 1 -- 172.21.15.97:0/3494967658 shutdown_connections 2024-03-29T04:08:31.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.047+0000 7fb299a74640 1 -- 172.21.15.97:0/3494967658 wait complete. 2024-03-29T04:08:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.048+0000 7fb299a74640 1 Processor -- start 2024-03-29T04:08:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.048+0000 7fb299a74640 1 -- start start 2024-03-29T04:08:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb299a74640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29406e1c0 0x7fb2941f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb299a74640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29406f1e0 0x7fb29415eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb299a74640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb294144790 0x7fb29415f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb299a74640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2940b3980 con 0x7fb29406f1e0 2024-03-29T04:08:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb299a74640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb2940b3800 con 0x7fb29406e1c0 2024-03-29T04:08:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb299a74640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb29406d930 con 0x7fb294144790 2024-03-29T04:08:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb292ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29406f1e0 0x7fb29415eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb292ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29406f1e0 0x7fb29415eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51178/0 (socket says 172.21.15.97:51178) 2024-03-29T04:08:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb292ffd640 1 -- 172.21.15.97:0/2185053298 learned_addr learned my addr 172.21.15.97:0/2185053298 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb2937fe640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29406e1c0 0x7fb2941f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb293fff640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb294144790 0x7fb29415f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb292ffd640 1 -- 172.21.15.97:0/2185053298 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb294144790 msgr2=0x7fb29415f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb292ffd640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb294144790 0x7fb29415f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb292ffd640 1 -- 172.21.15.97:0/2185053298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29406e1c0 msgr2=0x7fb2941f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb292ffd640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29406e1c0 0x7fb2941f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb292ffd640 1 -- 172.21.15.97:0/2185053298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2940aefb0 con 0x7fb29406f1e0 2024-03-29T04:08:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb2937fe640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29406e1c0 0x7fb2941f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb293fff640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb294144790 0x7fb29415f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb292ffd640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29406f1e0 0x7fb29415eac0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb288000c00 tx=0x7fb2880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb27bfff640 1 -- 172.21.15.97:0/2185053298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28802eae0 con 0x7fb29406f1e0 2024-03-29T04:08:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb27bfff640 1 -- 172.21.15.97:0/2185053298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2880375c0 con 0x7fb29406f1e0 2024-03-29T04:08:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb27bfff640 1 -- 172.21.15.97:0/2185053298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2880379b0 con 0x7fb29406f1e0 2024-03-29T04:08:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb299a74640 1 -- 172.21.15.97:0/2185053298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2940af290 con 0x7fb29406f1e0 2024-03-29T04:08:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.049+0000 7fb299a74640 1 -- 172.21.15.97:0/2185053298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2940af850 con 0x7fb29406f1e0 2024-03-29T04:08:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.050+0000 7fb299a74640 1 -- 172.21.15.97:0/2185053298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb294068640 con 0x7fb29406f1e0 2024-03-29T04:08:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.051+0000 7fb27bfff640 1 -- 172.21.15.97:0/2185053298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb288037b50 con 0x7fb29406f1e0 2024-03-29T04:08:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.051+0000 7fb27bfff640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb28006c530 0x7fb28006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.051+0000 7fb27bfff640 1 -- 172.21.15.97:0/2185053298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fb2880add00 con 0x7fb29406f1e0 2024-03-29T04:08:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.051+0000 7fb2937fe640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb28006c530 0x7fb28006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:31.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.052+0000 7fb2937fe640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb28006c530 0x7fb28006e9f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb284000f60 tx=0x7fb284009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:31.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.055+0000 7fb27bfff640 1 -- 172.21.15.97:0/2185053298 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb288037e40 con 0x7fb29406f1e0 2024-03-29T04:08:31.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.229+0000 7fb299a74640 1 -- 172.21.15.97:0/2185053298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2940cca10 con 0x7fb29406f1e0 2024-03-29T04:08:31.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.230+0000 7fb27bfff640 1 -- 172.21.15.97:0/2185053298 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16682 (secure 0 0 0) 0x7fb28807c330 con 0x7fb29406f1e0 2024-03-29T04:08:31.210 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:31.210 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":31,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:08:29.955551+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:08:28.403360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T04:08:28.909709+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.232+0000 7fb299a74640 1 -- 172.21.15.97:0/2185053298 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb28006c530 msgr2=0x7fb28006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.232+0000 7fb299a74640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb28006c530 0x7fb28006e9f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb284000f60 tx=0x7fb284009040 comp rx=0 tx=0).stop 2024-03-29T04:08:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.232+0000 7fb299a74640 1 -- 172.21.15.97:0/2185053298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29406f1e0 msgr2=0x7fb29415eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.232+0000 7fb299a74640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29406f1e0 0x7fb29415eac0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb288000c00 tx=0x7fb2880046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.232+0000 7fb299a74640 1 -- 172.21.15.97:0/2185053298 shutdown_connections 2024-03-29T04:08:31.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.232+0000 7fb299a74640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb28006c530 0x7fb28006e9f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.232+0000 7fb299a74640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb294144790 0x7fb29415f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.232+0000 7fb299a74640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29406f1e0 0x7fb29415eac0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.232+0000 7fb299a74640 1 --2- 172.21.15.97:0/2185053298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb29406e1c0 0x7fb2941f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.232+0000 7fb299a74640 1 -- 172.21.15.97:0/2185053298 >> 172.21.15.97:0/2185053298 conn(0x7fb29406b510 msgr2=0x7fb29415a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.232+0000 7fb299a74640 1 -- 172.21.15.97:0/2185053298 shutdown_connections 2024-03-29T04:08:31.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.232+0000 7fb299a74640 1 -- 172.21.15.97:0/2185053298 wait complete. 2024-03-29T04:08:31.224 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T04:08:31.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.347+0000 7f0bcb767640 1 Processor -- start 2024-03-29T04:08:31.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.348+0000 7f0bcb767640 1 -- start start 2024-03-29T04:08:31.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.349+0000 7f0bcb767640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0bc40d03e0 0x7f0bc4169620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:31.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.349+0000 7f0bcb767640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc40d2390 con 0x7f0bc40d1010 2024-03-29T04:08:31.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.349+0000 7f0bcb767640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc40d2670 con 0x7f0bc40d3430 2024-03-29T04:08:31.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.349+0000 7f0bcb767640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc40d5060 con 0x7f0bc40d03e0 2024-03-29T04:08:31.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.349+0000 7f0bc9503640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0bc40d1010 0x7f0bc40d13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55770/0 (socket says 172.21.15.97:55770) 2024-03-29T04:08:31.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.349+0000 7f0bc9503640 1 -- 172.21.15.97:0/1204936088 learned_addr learned my addr 172.21.15.97:0/1204936088 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:31.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.349+0000 7f0bc9d04640 1 --2- 172.21.15.97:0/1204936088 >> v2:172.21.15.97:3300/0 conn(0x7f0bc40d03e0 0x7f0bc4169620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:31.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.349+0000 7f0bc9d04640 1 -- 172.21.15.97:0/1204936088 >> v1:172.21.15.154:6789/0 conn(0x7f0bc40d3430 legacy=0x7f0bc40cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:31.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.349+0000 7f0bc9d04640 1 -- 172.21.15.97:0/1204936088 >> v1:172.21.15.97:6789/0 conn(0x7f0bc40d1010 legacy=0x7f0bc40d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:31.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.349+0000 7f0bc9d04640 1 -- 172.21.15.97:0/1204936088 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc40d2950 con 0x7f0bc40d03e0 2024-03-29T04:08:31.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.349+0000 7f0bbbfff640 1 -- 172.21.15.97:0/1204936088 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2581737294 0 0) 0x7f0bc40d2390 con 0x7f0bc40d1010 2024-03-29T04:08:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.350+0000 7f0bc9d04640 1 --2- 172.21.15.97:0/1204936088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40d03e0 0x7f0bc4169620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009d90 tx=0x7f0bbc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3dffc9efd410cd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.350+0000 7f0bbbfff640 1 -- 172.21.15.97:0/1204936088 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc039070 con 0x7f0bc40d03e0 2024-03-29T04:08:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.350+0000 7f0bbbfff640 1 -- 172.21.15.97:0/1204936088 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bbc02cc60 con 0x7f0bc40d03e0 2024-03-29T04:08:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.350+0000 7f0bbbfff640 1 -- 172.21.15.97:0/1204936088 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc034660 con 0x7f0bc40d03e0 2024-03-29T04:08:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.351+0000 7f0bcb767640 1 -- 172.21.15.97:0/1204936088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40d03e0 msgr2=0x7f0bc4169620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.351+0000 7f0bcb767640 1 --2- 172.21.15.97:0/1204936088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40d03e0 0x7f0bc4169620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009d90 tx=0x7f0bbc02beb0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.351+0000 7f0bcb767640 1 -- 172.21.15.97:0/1204936088 shutdown_connections 2024-03-29T04:08:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.351+0000 7f0bcb767640 1 --2- 172.21.15.97:0/1204936088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40d03e0 0x7f0bc4169620 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.351+0000 7f0bcb767640 1 -- 172.21.15.97:0/1204936088 >> 172.21.15.97:0/1204936088 conn(0x7f0bc408bfc0 msgr2=0x7f0bc40d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:31.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.351+0000 7f0bcb767640 1 -- 172.21.15.97:0/1204936088 shutdown_connections 2024-03-29T04:08:31.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.351+0000 7f0bcb767640 1 -- 172.21.15.97:0/1204936088 wait complete. 2024-03-29T04:08:31.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.352+0000 7f0bcb767640 1 Processor -- start 2024-03-29T04:08:31.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.352+0000 7f0bcb767640 1 -- start start 2024-03-29T04:08:31.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.352+0000 7f0bcb767640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc40d03e0 0x7f0bc40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:31.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.352+0000 7f0bcb767640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0bc40d1010 0x7f0bc4161170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:31.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.352+0000 7f0bcb767640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40d3430 0x7f0bc40b9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:31.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bcb767640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc415eaf0 con 0x7f0bc40d3430 2024-03-29T04:08:31.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bcb767640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc40d2f10 con 0x7f0bc40d03e0 2024-03-29T04:08:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bcb767640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc415ec70 con 0x7f0bc40d1010 2024-03-29T04:08:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bc9d04640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40d3430 0x7f0bc40b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bc9d04640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40d3430 0x7f0bc40b9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51194/0 (socket says 172.21.15.97:51194) 2024-03-29T04:08:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bc9d04640 1 -- 172.21.15.97:0/423516012 learned_addr learned my addr 172.21.15.97:0/423516012 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bc9503640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc40d03e0 0x7f0bc40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bc8d02640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0bc40d1010 0x7f0bc4161170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bc9d04640 1 -- 172.21.15.97:0/423516012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0bc40d1010 msgr2=0x7f0bc4161170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bc9d04640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0bc40d1010 0x7f0bc4161170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bc9d04640 1 -- 172.21.15.97:0/423516012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc40d03e0 msgr2=0x7f0bc40ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bc9d04640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc40d03e0 0x7f0bc40ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bc9d04640 1 -- 172.21.15.97:0/423516012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc415a9d0 con 0x7f0bc40d3430 2024-03-29T04:08:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bc9d04640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40d3430 0x7f0bc40b9530 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009d90 tx=0x7f0bbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bb9ffb640 1 -- 172.21.15.97:0/423516012 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc044020 con 0x7f0bc40d3430 2024-03-29T04:08:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bb9ffb640 1 -- 172.21.15.97:0/423516012 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bbc03f470 con 0x7f0bc40d3430 2024-03-29T04:08:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bcb767640 1 -- 172.21.15.97:0/423516012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc415acb0 con 0x7f0bc40d3430 2024-03-29T04:08:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bb9ffb640 1 -- 172.21.15.97:0/423516012 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc039040 con 0x7f0bc40d3430 2024-03-29T04:08:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.353+0000 7f0bcb767640 1 -- 172.21.15.97:0/423516012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc415b270 con 0x7f0bc40d3430 2024-03-29T04:08:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.354+0000 7f0bcb767640 1 -- 172.21.15.97:0/423516012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc40a2570 con 0x7f0bc40d3430 2024-03-29T04:08:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.355+0000 7f0bb9ffb640 1 -- 172.21.15.97:0/423516012 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0bbc047020 con 0x7f0bc40d3430 2024-03-29T04:08:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.355+0000 7f0bb9ffb640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0ba806c560 0x7f0ba806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.355+0000 7f0bb9ffb640 1 -- 172.21.15.97:0/423516012 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f0bbc0ae030 con 0x7f0bc40d3430 2024-03-29T04:08:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.355+0000 7f0bc9503640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0ba806c560 0x7f0ba806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.356+0000 7f0bc9503640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0ba806c560 0x7f0ba806ea20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4000f60 tx=0x7f0bb4007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:31.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.360+0000 7f0bb9ffb640 1 -- 172.21.15.97:0/423516012 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0bbc04e070 con 0x7f0bc40d3430 2024-03-29T04:08:31.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:31.579+0000 7f0bcb767640 1 -- 172.21.15.97:0/423516012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0bc404f9d0 con 0x7f0bc40d3430 2024-03-29T04:08:32.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.238+0000 7f0bb9ffb640 1 -- 172.21.15.97:0/423516012 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) ==== 118+0+0 (secure 0 0 0) 0x7f0bbc07c6e0 con 0x7f0bc40d3430 2024-03-29T04:08:32.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.241+0000 7f0bcb767640 1 -- 172.21.15.97:0/423516012 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0ba806c560 msgr2=0x7f0ba806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:32.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.241+0000 7f0bcb767640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0ba806c560 0x7f0ba806ea20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4000f60 tx=0x7f0bb4007b50 comp rx=0 tx=0).stop 2024-03-29T04:08:32.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.241+0000 7f0bcb767640 1 -- 172.21.15.97:0/423516012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40d3430 msgr2=0x7f0bc40b9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:32.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.241+0000 7f0bcb767640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40d3430 0x7f0bc40b9530 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009d90 tx=0x7f0bbc004a40 comp rx=0 tx=0).stop 2024-03-29T04:08:32.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.241+0000 7f0bcb767640 1 -- 172.21.15.97:0/423516012 shutdown_connections 2024-03-29T04:08:32.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.241+0000 7f0bcb767640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0ba806c560 0x7f0ba806ea20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:32.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.241+0000 7f0bcb767640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bc40d3430 0x7f0bc40b9530 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:32.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.241+0000 7f0bcb767640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0bc40d1010 0x7f0bc4161170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:32.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.241+0000 7f0bcb767640 1 --2- 172.21.15.97:0/423516012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc40d03e0 0x7f0bc40ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:32.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.241+0000 7f0bcb767640 1 -- 172.21.15.97:0/423516012 >> 172.21.15.97:0/423516012 conn(0x7f0bc408bfc0 msgr2=0x7f0bc40d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:32.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.241+0000 7f0bcb767640 1 -- 172.21.15.97:0/423516012 shutdown_connections 2024-03-29T04:08:32.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.241+0000 7f0bcb767640 1 -- 172.21.15.97:0/423516012 wait complete. 2024-03-29T04:08:32.232 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T04:08:32.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T04:08:32.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.357+0000 7fd7a3160640 1 Processor -- start 2024-03-29T04:08:32.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.357+0000 7fd7a3160640 1 -- start start 2024-03-29T04:08:32.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.358+0000 7fd7a3160640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd79c0d16a0 0x7fd79c0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:32.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.358+0000 7fd7a3160640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd79c15ed00 con 0x7fd79c15e920 2024-03-29T04:08:32.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.358+0000 7fd7a3160640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd79c0d21e0 con 0x7fd79c0cfde0 2024-03-29T04:08:32.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.358+0000 7fd7a3160640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fd79c0d2480 con 0x7fd79c0d16a0 2024-03-29T04:08:32.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.358+0000 7fd7a16fd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd79c0d16a0 0x7fd79c0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:32.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.358+0000 7fd7a0efc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd79c15e920 0x7fd79c0d3240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55780/0 (socket says 172.21.15.97:55780) 2024-03-29T04:08:32.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.358+0000 7fd7a0efc640 1 -- 172.21.15.97:0/1471009644 learned_addr learned my addr 172.21.15.97:0/1471009644 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:32.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.359+0000 7fd7a16fd640 1 -- 172.21.15.97:0/1471009644 >> v1:172.21.15.154:6789/0 conn(0x7fd79c0cfde0 legacy=0x7fd79c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:32.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.359+0000 7fd7a16fd640 1 -- 172.21.15.97:0/1471009644 >> v1:172.21.15.97:6789/0 conn(0x7fd79c15e920 legacy=0x7fd79c0d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:32.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.359+0000 7fd7a16fd640 1 -- 172.21.15.97:0/1471009644 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c0d2760 con 0x7fd79c0d16a0 2024-03-29T04:08:32.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.359+0000 7fd79b7fe640 1 -- 172.21.15.97:0/1471009644 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 821921730 0 0) 0x7fd79c15ed00 con 0x7fd79c15e920 2024-03-29T04:08:32.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.359+0000 7fd7a16fd640 1 --2- 172.21.15.97:0/1471009644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd79c0d16a0 0x7fd79c0d0240 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd794009d90 tx=0x7fd79402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b73b8a566f195a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:32.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.360+0000 7fd79b7fe640 1 -- 172.21.15.97:0/1471009644 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794039070 con 0x7fd79c0d16a0 2024-03-29T04:08:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.360+0000 7fd79b7fe640 1 -- 172.21.15.97:0/1471009644 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd79402cd00 con 0x7fd79c0d16a0 2024-03-29T04:08:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.360+0000 7fd79b7fe640 1 -- 172.21.15.97:0/1471009644 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794034660 con 0x7fd79c0d16a0 2024-03-29T04:08:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.360+0000 7fd7a3160640 1 -- 172.21.15.97:0/1471009644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd79c0d16a0 msgr2=0x7fd79c0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.360+0000 7fd7a3160640 1 --2- 172.21.15.97:0/1471009644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd79c0d16a0 0x7fd79c0d0240 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd794009d90 tx=0x7fd79402bf50 comp rx=0 tx=0).stop 2024-03-29T04:08:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.360+0000 7fd7a3160640 1 -- 172.21.15.97:0/1471009644 shutdown_connections 2024-03-29T04:08:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.360+0000 7fd7a3160640 1 --2- 172.21.15.97:0/1471009644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd79c0d16a0 0x7fd79c0d0240 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.360+0000 7fd7a3160640 1 -- 172.21.15.97:0/1471009644 >> 172.21.15.97:0/1471009644 conn(0x7fd79c08bfc0 msgr2=0x7fd79c0d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.360+0000 7fd7a3160640 1 -- 172.21.15.97:0/1471009644 shutdown_connections 2024-03-29T04:08:32.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.360+0000 7fd7a3160640 1 -- 172.21.15.97:0/1471009644 wait complete. 2024-03-29T04:08:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.362+0000 7fd7a3160640 1 Processor -- start 2024-03-29T04:08:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.362+0000 7fd7a3160640 1 -- start start 2024-03-29T04:08:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.362+0000 7fd7a3160640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd79c0cfde0 0x7fd79c15cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.362+0000 7fd7a3160640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd79c0d16a0 0x7fd79c15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.362+0000 7fd7a3160640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd79c15e920 0x7fd79c15b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a3160640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c0d4ef0 con 0x7fd79c15e920 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a3160640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c0d2d20 con 0x7fd79c0d16a0 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a3160640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd79c0d5070 con 0x7fd79c0cfde0 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a16fd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd79c15e920 0x7fd79c15b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a16fd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd79c15e920 0x7fd79c15b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51206/0 (socket says 172.21.15.97:51206) 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a16fd640 1 -- 172.21.15.97:0/919437862 learned_addr learned my addr 172.21.15.97:0/919437862 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a0efc640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd79c0cfde0 0x7fd79c15cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd79bfff640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd79c0d16a0 0x7fd79c15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a16fd640 1 -- 172.21.15.97:0/919437862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd79c0cfde0 msgr2=0x7fd79c15cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a16fd640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd79c0cfde0 0x7fd79c15cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a16fd640 1 -- 172.21.15.97:0/919437862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd79c0d16a0 msgr2=0x7fd79c15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a16fd640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd79c0d16a0 0x7fd79c15eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a16fd640 1 -- 172.21.15.97:0/919437862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c1586f0 con 0x7fd79c15e920 2024-03-29T04:08:32.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a0efc640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd79c0cfde0 0x7fd79c15cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:32.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a16fd640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd79c15e920 0x7fd79c15b590 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd794037040 tx=0x7fd794004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:32.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7997fa640 1 -- 172.21.15.97:0/919437862 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794044030 con 0x7fd79c15e920 2024-03-29T04:08:32.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7997fa640 1 -- 172.21.15.97:0/919437862 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd79403f470 con 0x7fd79c15e920 2024-03-29T04:08:32.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a3160640 1 -- 172.21.15.97:0/919437862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79c15c280 con 0x7fd79c15e920 2024-03-29T04:08:32.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7997fa640 1 -- 172.21.15.97:0/919437862 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794039040 con 0x7fd79c15e920 2024-03-29T04:08:32.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.363+0000 7fd7a3160640 1 -- 172.21.15.97:0/919437862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79c158c70 con 0x7fd79c15e920 2024-03-29T04:08:32.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.364+0000 7fd7a3160640 1 -- 172.21.15.97:0/919437862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79c0a2560 con 0x7fd79c15e920 2024-03-29T04:08:32.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.365+0000 7fd7997fa640 1 -- 172.21.15.97:0/919437862 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd794047070 con 0x7fd79c15e920 2024-03-29T04:08:32.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.365+0000 7fd7997fa640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd78006c530 0x7fd78006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:32.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.365+0000 7fd7997fa640 1 -- 172.21.15.97:0/919437862 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd7940ae0f0 con 0x7fd79c15e920 2024-03-29T04:08:32.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.365+0000 7fd7a0efc640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd78006c530 0x7fd78006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:32.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.366+0000 7fd7a0efc640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd78006c530 0x7fd78006e9f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd790007fd0 tx=0x7fd790007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:32.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.370+0000 7fd7997fa640 1 -- 172.21.15.97:0/919437862 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd79404e070 con 0x7fd79c15e920 2024-03-29T04:08:32.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:32.583+0000 7fd7a3160640 1 -- 172.21.15.97:0/919437862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd79c04f9d0 con 0x7fd79c15e920 2024-03-29T04:08:33.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.239+0000 7fd7997fa640 1 -- 172.21.15.97:0/919437862 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fd79407c7b0 con 0x7fd79c15e920 2024-03-29T04:08:33.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.242+0000 7fd7a3160640 1 -- 172.21.15.97:0/919437862 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd78006c530 msgr2=0x7fd78006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.242+0000 7fd7a3160640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd78006c530 0x7fd78006e9f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd790007fd0 tx=0x7fd790007dc0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.242+0000 7fd7a3160640 1 -- 172.21.15.97:0/919437862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd79c15e920 msgr2=0x7fd79c15b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.242+0000 7fd7a3160640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd79c15e920 0x7fd79c15b590 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd794037040 tx=0x7fd794004a40 comp rx=0 tx=0).stop 2024-03-29T04:08:33.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.242+0000 7fd7a3160640 1 -- 172.21.15.97:0/919437862 shutdown_connections 2024-03-29T04:08:33.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.242+0000 7fd7a3160640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd78006c530 0x7fd78006e9f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.242+0000 7fd7a3160640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd79c15e920 0x7fd79c15b590 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.242+0000 7fd7a3160640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd79c0d16a0 0x7fd79c15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.242+0000 7fd7a3160640 1 --2- 172.21.15.97:0/919437862 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd79c0cfde0 0x7fd79c15cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.242+0000 7fd7a3160640 1 -- 172.21.15.97:0/919437862 >> 172.21.15.97:0/919437862 conn(0x7fd79c08bfc0 msgr2=0x7fd79c0cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:33.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.242+0000 7fd7a3160640 1 -- 172.21.15.97:0/919437862 shutdown_connections 2024-03-29T04:08:33.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.242+0000 7fd7a3160640 1 -- 172.21.15.97:0/919437862 wait complete. 2024-03-29T04:08:33.232 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T04:08:33.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:08:33.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.354+0000 7ff1d518d640 1 Processor -- start 2024-03-29T04:08:33.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.354+0000 7ff1d518d640 1 -- start start 2024-03-29T04:08:33.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.356+0000 7ff1d518d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff1d00b3c50 0x7ff1d00d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.356+0000 7ff1d518d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1d00d3140 0x7ff1d006cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.356+0000 7ff1d518d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1d00cfb30 con 0x7ff1d015e930 2024-03-29T04:08:33.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.356+0000 7ff1d518d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ff1d006c780 con 0x7ff1d00b3c50 2024-03-29T04:08:33.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.356+0000 7ff1d518d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d00cfe10 con 0x7ff1d00d3140 2024-03-29T04:08:33.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.356+0000 7ff1ce59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff1d00b3c50 0x7ff1d00d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.356+0000 7ff1cf59e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1d00d3140 0x7ff1d006cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.356+0000 7ff1ce59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff1d00b3c50 0x7ff1d00d2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:57338/0 (socket says 172.21.15.97:57338) 2024-03-29T04:08:33.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.356+0000 7ff1ced9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff1d015e930 0x7ff1d00cf130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:53242/0 (socket says 172.21.15.97:53242) 2024-03-29T04:08:33.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.356+0000 7ff1ce59c640 1 -- 172.21.15.97:0/2080298872 learned_addr learned my addr 172.21.15.97:0/2080298872 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:33.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.357+0000 7ff1ce59c640 1 -- 172.21.15.97:0/2080298872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1d00d3140 msgr2=0x7ff1d006cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:33.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.357+0000 7ff1ce59c640 1 --2- 172.21.15.97:0/2080298872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1d00d3140 0x7ff1d006cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.357+0000 7ff1ce59c640 1 -- 172.21.15.97:0/2080298872 >> v1:172.21.15.154:6789/0 conn(0x7ff1d015e930 legacy=0x7ff1d00cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.357+0000 7ff1cdd9b640 1 -- 172.21.15.97:0/2080298872 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3383122 0 0) 0x7ff1d00cfb30 con 0x7ff1d015e930 2024-03-29T04:08:33.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.357+0000 7ff1ce59c640 1 -- 172.21.15.97:0/2080298872 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d00d00f0 con 0x7ff1d00b3c50 2024-03-29T04:08:33.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.357+0000 7ff1cf59e640 1 --2- 172.21.15.97:0/2080298872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1d00d3140 0x7ff1d006cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:33.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.357+0000 7ff1ce59c640 1 --2- 172.21.15.97:0/2080298872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d00b3c50 0x7ff1d00d2c00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009870 tx=0x7ff1c002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=811924f32f2f5735 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:33.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.358+0000 7ff1cdd9b640 1 -- 172.21.15.97:0/2080298872 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0035020 con 0x7ff1d00b3c50 2024-03-29T04:08:33.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.358+0000 7ff1cdd9b640 1 -- 172.21.15.97:0/2080298872 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c0031640 con 0x7ff1d00b3c50 2024-03-29T04:08:33.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.358+0000 7ff1cdd9b640 1 -- 172.21.15.97:0/2080298872 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c002d040 con 0x7ff1d00b3c50 2024-03-29T04:08:33.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.358+0000 7ff1d518d640 1 -- 172.21.15.97:0/2080298872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d00b3c50 msgr2=0x7ff1d00d2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.358+0000 7ff1d518d640 1 --2- 172.21.15.97:0/2080298872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d00b3c50 0x7ff1d00d2c00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009870 tx=0x7ff1c002b8f0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.358+0000 7ff1d518d640 1 -- 172.21.15.97:0/2080298872 shutdown_connections 2024-03-29T04:08:33.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.358+0000 7ff1d518d640 1 --2- 172.21.15.97:0/2080298872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1d00d3140 0x7ff1d006cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.358+0000 7ff1d518d640 1 --2- 172.21.15.97:0/2080298872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d00b3c50 0x7ff1d00d2c00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.358+0000 7ff1d518d640 1 -- 172.21.15.97:0/2080298872 >> 172.21.15.97:0/2080298872 conn(0x7ff1d008bfc0 msgr2=0x7ff1d00d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:33.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.358+0000 7ff1d518d640 1 -- 172.21.15.97:0/2080298872 shutdown_connections 2024-03-29T04:08:33.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.358+0000 7ff1d518d640 1 -- 172.21.15.97:0/2080298872 wait complete. 2024-03-29T04:08:33.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.360+0000 7ff1d518d640 1 Processor -- start 2024-03-29T04:08:33.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.360+0000 7ff1d518d640 1 -- start start 2024-03-29T04:08:33.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.360+0000 7ff1d518d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1d00b3c50 0x7ff1d01ee720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.360+0000 7ff1d518d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1d00d3140 0x7ff1d01f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.360+0000 7ff1d518d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d015e930 0x7ff1d015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.360+0000 7ff1d518d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d015ed10 con 0x7ff1d00b3c50 2024-03-29T04:08:33.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.360+0000 7ff1d518d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d00cf840 con 0x7ff1d015e930 2024-03-29T04:08:33.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.360+0000 7ff1d518d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d015ee90 con 0x7ff1d00d3140 2024-03-29T04:08:33.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.360+0000 7ff1ced9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1d00b3c50 0x7ff1d01ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.360+0000 7ff1ced9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1d00b3c50 0x7ff1d01ee720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51216/0 (socket says 172.21.15.97:51216) 2024-03-29T04:08:33.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.360+0000 7ff1ced9d640 1 -- 172.21.15.97:0/4044723106 learned_addr learned my addr 172.21.15.97:0/4044723106 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:33.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.361+0000 7ff1ce59c640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1d00d3140 0x7ff1d01f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.361+0000 7ff1cf59e640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d015e930 0x7ff1d015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.361+0000 7ff1ced9d640 1 -- 172.21.15.97:0/4044723106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1d00d3140 msgr2=0x7ff1d01f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.361+0000 7ff1ced9d640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1d00d3140 0x7ff1d01f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.361+0000 7ff1ced9d640 1 -- 172.21.15.97:0/4044723106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d015e930 msgr2=0x7ff1d015c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.361+0000 7ff1ced9d640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d015e930 0x7ff1d015c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.361+0000 7ff1ced9d640 1 -- 172.21.15.97:0/4044723106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d01f5660 con 0x7ff1d00b3c50 2024-03-29T04:08:33.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.361+0000 7ff1ced9d640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1d00b3c50 0x7ff1d01ee720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800c4d0 tx=0x7ff1b800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:33.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.361+0000 7ff1c77fe640 1 -- 172.21.15.97:0/4044723106 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80074e0 con 0x7ff1d00b3c50 2024-03-29T04:08:33.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.361+0000 7ff1c77fe640 1 -- 172.21.15.97:0/4044723106 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b80047e0 con 0x7ff1d00b3c50 2024-03-29T04:08:33.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.361+0000 7ff1d518d640 1 -- 172.21.15.97:0/4044723106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d00cba20 con 0x7ff1d00b3c50 2024-03-29T04:08:33.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.361+0000 7ff1c77fe640 1 -- 172.21.15.97:0/4044723106 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8004bd0 con 0x7ff1d00b3c50 2024-03-29T04:08:33.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.361+0000 7ff1d518d640 1 -- 172.21.15.97:0/4044723106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d00cbfe0 con 0x7ff1d00b3c50 2024-03-29T04:08:33.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.362+0000 7ff1d518d640 1 -- 172.21.15.97:0/4044723106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d00d2220 con 0x7ff1d00b3c50 2024-03-29T04:08:33.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.363+0000 7ff1c77fe640 1 -- 172.21.15.97:0/4044723106 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff1b8011070 con 0x7ff1d00b3c50 2024-03-29T04:08:33.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.363+0000 7ff1c77fe640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff1b006c560 0x7ff1b006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.363+0000 7ff1ce59c640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff1b006c560 0x7ff1b006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.363+0000 7ff1c77fe640 1 -- 172.21.15.97:0/4044723106 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff1b808b200 con 0x7ff1d00b3c50 2024-03-29T04:08:33.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.364+0000 7ff1ce59c640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff1b006c560 0x7ff1b006ea20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0036000 tx=0x7ff1c003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:33.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.367+0000 7ff1c77fe640 1 -- 172.21.15.97:0/4044723106 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff1d00d2220 con 0x7ff1d00b3c50 2024-03-29T04:08:33.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.586+0000 7ff1d518d640 1 -- 172.21.15.97:0/4044723106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1d0068640 con 0x7ff1d00b3c50 2024-03-29T04:08:33.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.587+0000 7ff1c77fe640 1 -- 172.21.15.97:0/4044723106 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5184 (secure 0 0 0) 0x7ff1d01f5660 con 0x7ff1d00b3c50 2024-03-29T04:08:33.567 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:33.567 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2199898540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2199898540},{"type":"v1","addr":"172.21.15.97:6837","nonce":2199898540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1223352413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1223352413},{"type":"v1","addr":"172.21.15.154:6834","nonce":1223352413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/830866132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":830866132},{"type":"v1","addr":"172.21.15.154:6835","nonce":830866132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:08:29.955858+0000","modified":"2024-03-29T04:08:32.584967+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5067":{"gid":5067,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/3142898446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3142898446},{"type":"v1","addr":"172.21.15.97:6835","nonce":3142898446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5067,"qdb_cluster":[5067]},"id":2}]} 2024-03-29T04:08:33.567 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 14 2024-03-29T04:08:33.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.589+0000 7ff1d518d640 1 -- 172.21.15.97:0/4044723106 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff1b006c560 msgr2=0x7ff1b006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.589+0000 7ff1d518d640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff1b006c560 0x7ff1b006ea20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0036000 tx=0x7ff1c003c040 comp rx=0 tx=0).stop 2024-03-29T04:08:33.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.589+0000 7ff1d518d640 1 -- 172.21.15.97:0/4044723106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1d00b3c50 msgr2=0x7ff1d01ee720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.589+0000 7ff1d518d640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1d00b3c50 0x7ff1d01ee720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800c4d0 tx=0x7ff1b800c8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.589+0000 7ff1d518d640 1 -- 172.21.15.97:0/4044723106 shutdown_connections 2024-03-29T04:08:33.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.589+0000 7ff1d518d640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff1b006c560 0x7ff1b006ea20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.589+0000 7ff1d518d640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d015e930 0x7ff1d015c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.589+0000 7ff1d518d640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1d00d3140 0x7ff1d01f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.589+0000 7ff1d518d640 1 --2- 172.21.15.97:0/4044723106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1d00b3c50 0x7ff1d01ee720 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.589+0000 7ff1d518d640 1 -- 172.21.15.97:0/4044723106 >> 172.21.15.97:0/4044723106 conn(0x7ff1d008bfc0 msgr2=0x7ff1d00b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:33.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.589+0000 7ff1d518d640 1 -- 172.21.15.97:0/4044723106 shutdown_connections 2024-03-29T04:08:33.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.589+0000 7ff1d518d640 1 -- 172.21.15.97:0/4044723106 wait complete. 2024-03-29T04:08:33.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:33.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.705+0000 7fd72bdd4640 1 Processor -- start 2024-03-29T04:08:33.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.705+0000 7fd72bdd4640 1 -- start start 2024-03-29T04:08:33.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd72bdd4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7240ae1b0 0x7fd72406b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd72bdd4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7240b41d0 con 0x7fd72415e870 2024-03-29T04:08:33.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd72bdd4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7240b4470 con 0x7fd7240ae630 2024-03-29T04:08:33.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd72bdd4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd7240b6510 con 0x7fd7240ae1b0 2024-03-29T04:08:33.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd729b70640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd72415e870 0x7fd7240b26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55794/0 (socket says 172.21.15.97:55794) 2024-03-29T04:08:33.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd729b70640 1 -- 172.21.15.97:0/496080687 learned_addr learned my addr 172.21.15.97:0/496080687 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:33.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd72936f640 1 --2- 172.21.15.97:0/496080687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7240ae1b0 0x7fd72406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3403845266 0 0) 0x7fd7240b41d0 con 0x7fd72415e870 2024-03-29T04:08:33.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd710003520 con 0x7fd72415e870 2024-03-29T04:08:33.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3076743755 0 0) 0x7fd710003520 con 0x7fd72415e870 2024-03-29T04:08:33.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7240ae1b0 msgr2=0x7fd72406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:33.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd728b6e640 1 --2- 172.21.15.97:0/496080687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7240ae1b0 0x7fd72406b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 >> v1:172.21.15.154:6789/0 conn(0x7fd7240ae630 legacy=0x7fd7240b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.706+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7240b4750 con 0x7fd72415e870 2024-03-29T04:08:33.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7fd720001690 con 0x7fd72415e870 2024-03-29T04:08:33.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 >> v1:172.21.15.97:6789/0 conn(0x7fd72415e870 legacy=0x7fd7240b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd728b6e640 1 --2- 172.21.15.97:0/496080687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7100039e0 0x7fd710003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd728b6e640 1 --2- 172.21.15.97:0/496080687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd710004390 0x7fd710004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd728b6e640 1 --2- 172.21.15.97:0/496080687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7240ae1b0 0x7fd710005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7240b6510 con 0x7fd710004390 2024-03-29T04:08:33.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd71000ba70 con 0x7fd7100039e0 2024-03-29T04:08:33.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd71000bc70 con 0x7fd7240ae1b0 2024-03-29T04:08:33.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd72936f640 1 --2- 172.21.15.97:0/496080687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd710004390 0x7fd710004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd729b70640 1 --2- 172.21.15.97:0/496080687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7100039e0 0x7fd710003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd72a371640 1 --2- 172.21.15.97:0/496080687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7240ae1b0 0x7fd710005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd72936f640 1 -- 172.21.15.97:0/496080687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7240ae1b0 msgr2=0x7fd710005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:33.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd72936f640 1 --2- 172.21.15.97:0/496080687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7240ae1b0 0x7fd710005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd72936f640 1 -- 172.21.15.97:0/496080687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7100039e0 msgr2=0x7fd710003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:33.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd72936f640 1 --2- 172.21.15.97:0/496080687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7100039e0 0x7fd710003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd72936f640 1 -- 172.21.15.97:0/496080687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd710003520 con 0x7fd710004390 2024-03-29T04:08:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.707+0000 7fd729b70640 1 --2- 172.21.15.97:0/496080687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7100039e0 0x7fd710003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.708+0000 7fd72936f640 1 --2- 172.21.15.97:0/496080687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd710004390 0x7fd710004860 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd714009ac0 tx=0x7fd710005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df29b46e596628d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.708+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714039070 con 0x7fd710004390 2024-03-29T04:08:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.708+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd71402c6b0 con 0x7fd710004390 2024-03-29T04:08:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.708+0000 7fd728b6e640 1 -- 172.21.15.97:0/496080687 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714009690 con 0x7fd710004390 2024-03-29T04:08:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.708+0000 7fd72bdd4640 1 -- 172.21.15.97:0/496080687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd710004390 msgr2=0x7fd710004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.708+0000 7fd72bdd4640 1 --2- 172.21.15.97:0/496080687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd710004390 0x7fd710004860 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd714009ac0 tx=0x7fd710005830 comp rx=0 tx=0).stop 2024-03-29T04:08:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.708+0000 7fd72a371640 1 -- 172.21.15.97:0/496080687 reap_dead start 2024-03-29T04:08:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.708+0000 7fd72bdd4640 1 -- 172.21.15.97:0/496080687 shutdown_connections 2024-03-29T04:08:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.708+0000 7fd72bdd4640 1 -- 172.21.15.97:0/496080687 >> 172.21.15.97:0/496080687 conn(0x7fd7240b66f0 msgr2=0x7fd7240b6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.708+0000 7fd72bdd4640 1 -- 172.21.15.97:0/496080687 shutdown_connections 2024-03-29T04:08:33.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.709+0000 7fd72bdd4640 1 -- 172.21.15.97:0/496080687 wait complete. 2024-03-29T04:08:33.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.710+0000 7fd72bdd4640 1 Processor -- start 2024-03-29T04:08:33.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.710+0000 7fd72bdd4640 1 -- start start 2024-03-29T04:08:33.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.710+0000 7fd72bdd4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd710004390 0x7fd7241471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.710+0000 7fd72bdd4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd724148480 0x7fd724144f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.710+0000 7fd72bdd4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd724145850 0x7fd7241478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.710+0000 7fd72bdd4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7240b04c0 con 0x7fd724145850 2024-03-29T04:08:33.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.710+0000 7fd72bdd4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd72415ec50 con 0x7fd710004390 2024-03-29T04:08:33.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.710+0000 7fd72bdd4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd7240b5d50 con 0x7fd724148480 2024-03-29T04:08:33.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.710+0000 7fd72a371640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd724145850 0x7fd7241478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.710+0000 7fd72a371640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd724145850 0x7fd7241478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51238/0 (socket says 172.21.15.97:51238) 2024-03-29T04:08:33.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.710+0000 7fd72a371640 1 -- 172.21.15.97:0/3392515451 learned_addr learned my addr 172.21.15.97:0/3392515451 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:33.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.711+0000 7fd72936f640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd724148480 0x7fd724144f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.711+0000 7fd729b70640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd710004390 0x7fd7241471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.711+0000 7fd72a371640 1 -- 172.21.15.97:0/3392515451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd724148480 msgr2=0x7fd724144f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.711+0000 7fd72a371640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd724148480 0x7fd724144f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.711+0000 7fd72a371640 1 -- 172.21.15.97:0/3392515451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd710004390 msgr2=0x7fd7241471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.711+0000 7fd72a371640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd710004390 0x7fd7241471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.711+0000 7fd72a371640 1 -- 172.21.15.97:0/3392515451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd724146150 con 0x7fd724145850 2024-03-29T04:08:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.711+0000 7fd72a371640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd724145850 0x7fd7241478e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd724072e20 tx=0x7fd71c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.711+0000 7fd71a7fc640 1 -- 172.21.15.97:0/3392515451 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c003970 con 0x7fd724145850 2024-03-29T04:08:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.711+0000 7fd71a7fc640 1 -- 172.21.15.97:0/3392515451 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd71c018070 con 0x7fd724145850 2024-03-29T04:08:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.711+0000 7fd72bdd4640 1 -- 172.21.15.97:0/3392515451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd724146430 con 0x7fd724145850 2024-03-29T04:08:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.711+0000 7fd71a7fc640 1 -- 172.21.15.97:0/3392515451 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c003d50 con 0x7fd724145850 2024-03-29T04:08:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.711+0000 7fd72bdd4640 1 -- 172.21.15.97:0/3392515451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7241469f0 con 0x7fd724145850 2024-03-29T04:08:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.712+0000 7fd72bdd4640 1 -- 172.21.15.97:0/3392515451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd724068830 con 0x7fd724145850 2024-03-29T04:08:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.712+0000 7fd71a7fc640 1 -- 172.21.15.97:0/3392515451 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd71c0050d0 con 0x7fd724145850 2024-03-29T04:08:33.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.713+0000 7fd71a7fc640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd70806c560 0x7fd70806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:33.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.713+0000 7fd729b70640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd70806c560 0x7fd70806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:33.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.713+0000 7fd71a7fc640 1 -- 172.21.15.97:0/3392515451 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd71c08b480 con 0x7fd724145850 2024-03-29T04:08:33.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.713+0000 7fd729b70640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd70806c560 0x7fd70806ea20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd7200010e0 tx=0x7fd720000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:33.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.717+0000 7fd71a7fc640 1 -- 172.21.15.97:0/3392515451 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd724068830 con 0x7fd724145850 2024-03-29T04:08:33.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.899+0000 7fd72bdd4640 1 -- 172.21.15.97:0/3392515451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd72406f280 con 0x7fd724145850 2024-03-29T04:08:33.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.900+0000 7fd71a7fc640 1 -- 172.21.15.97:0/3392515451 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16669 (secure 0 0 0) 0x7fd71c059aa0 con 0x7fd724145850 2024-03-29T04:08:33.880 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:33.880 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":32,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:08:31.615419+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:08:28.403360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T04:08:28.909709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.903+0000 7fd72bdd4640 1 -- 172.21.15.97:0/3392515451 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd70806c560 msgr2=0x7fd70806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.903+0000 7fd72bdd4640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd70806c560 0x7fd70806ea20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd7200010e0 tx=0x7fd720000ed0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.903+0000 7fd72bdd4640 1 -- 172.21.15.97:0/3392515451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd724145850 msgr2=0x7fd7241478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.903+0000 7fd72bdd4640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd724145850 0x7fd7241478e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd724072e20 tx=0x7fd71c00eb90 comp rx=0 tx=0).stop 2024-03-29T04:08:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.903+0000 7fd72bdd4640 1 -- 172.21.15.97:0/3392515451 shutdown_connections 2024-03-29T04:08:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.903+0000 7fd72bdd4640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd70806c560 0x7fd70806ea20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.903+0000 7fd72bdd4640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd724145850 0x7fd7241478e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.903+0000 7fd72bdd4640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd724148480 0x7fd724144f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.903+0000 7fd72bdd4640 1 --2- 172.21.15.97:0/3392515451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd710004390 0x7fd7241471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.903+0000 7fd72bdd4640 1 -- 172.21.15.97:0/3392515451 >> 172.21.15.97:0/3392515451 conn(0x7fd7240b66f0 msgr2=0x7fd7240b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.903+0000 7fd72bdd4640 1 -- 172.21.15.97:0/3392515451 shutdown_connections 2024-03-29T04:08:33.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:33.903+0000 7fd72bdd4640 1 -- 172.21.15.97:0/3392515451 wait complete. 2024-03-29T04:08:33.897 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T04:08:33.898 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:08:34.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.027+0000 7fd04ac9a640 1 Processor -- start 2024-03-29T04:08:34.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.027+0000 7fd04ac9a640 1 -- start start 2024-03-29T04:08:34.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd04ac9a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd044144790 0x7fd0440b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd04ac9a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0440b3c10 con 0x7fd04406f1e0 2024-03-29T04:08:34.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd04ac9a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0440b3ef0 con 0x7fd04406e1c0 2024-03-29T04:08:34.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd04ac9a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fd0440b39d0 con 0x7fd044144790 2024-03-29T04:08:34.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd043fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd04406f1e0 0x7fd0440b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55808/0 (socket says 172.21.15.97:55808) 2024-03-29T04:08:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd043fff640 1 -- 172.21.15.97:0/3829499063 learned_addr learned my addr 172.21.15.97:0/3829499063 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 536140215 0 0) 0x7fd0440b3c10 con 0x7fd04406f1e0 2024-03-29T04:08:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd048a36640 1 --2- 172.21.15.97:0/3829499063 >> v2:172.21.15.154:3300/0 conn(0x7fd044144790 0x7fd0440b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd034004630 con 0x7fd04406f1e0 2024-03-29T04:08:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1325753545 0 0) 0x7fd034004630 con 0x7fd04406f1e0 2024-03-29T04:08:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 >> v2:172.21.15.154:3300/0 conn(0x7fd044144790 msgr2=0x7fd0440b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 --2- 172.21.15.97:0/3829499063 >> v2:172.21.15.154:3300/0 conn(0x7fd044144790 0x7fd0440b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 >> v1:172.21.15.154:6789/0 conn(0x7fd04406e1c0 legacy=0x7fd0440ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0440b41d0 con 0x7fd04406f1e0 2024-03-29T04:08:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7fd030001670 con 0x7fd04406f1e0 2024-03-29T04:08:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 >> v1:172.21.15.97:6789/0 conn(0x7fd04406f1e0 legacy=0x7fd0440b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 --2- 172.21.15.97:0/3829499063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd034004ac0 0x7fd034004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 --2- 172.21.15.97:0/3829499063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd034005910 0x7fd034005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 --2- 172.21.15.97:0/3829499063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0340062d0 0x7fd03400cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd0440b39d0 con 0x7fd034004ac0 2024-03-29T04:08:34.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd03400d170 con 0x7fd034005910 2024-03-29T04:08:34.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd03400d370 con 0x7fd0340062d0 2024-03-29T04:08:34.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.028+0000 7fd048a36640 1 --2- 172.21.15.97:0/3829499063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd034004ac0 0x7fd034004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.029+0000 7fd043fff640 1 --2- 172.21.15.97:0/3829499063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd034005910 0x7fd034005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.029+0000 7fd049237640 1 --2- 172.21.15.97:0/3829499063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0340062d0 0x7fd03400cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.029+0000 7fd048a36640 1 -- 172.21.15.97:0/3829499063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0340062d0 msgr2=0x7fd03400cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:34.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.029+0000 7fd048a36640 1 --2- 172.21.15.97:0/3829499063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0340062d0 0x7fd03400cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.029+0000 7fd048a36640 1 -- 172.21.15.97:0/3829499063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd034005910 msgr2=0x7fd034005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:34.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.029+0000 7fd048a36640 1 --2- 172.21.15.97:0/3829499063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd034005910 0x7fd034005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.029+0000 7fd048a36640 1 -- 172.21.15.97:0/3829499063 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd034004630 con 0x7fd034004ac0 2024-03-29T04:08:34.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.029+0000 7fd049237640 1 --2- 172.21.15.97:0/3829499063 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd0340062d0 0x7fd03400cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:34.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.029+0000 7fd049237640 1 -- 172.21.15.97:0/3829499063 reap_dead start 2024-03-29T04:08:34.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.030+0000 7fd048a36640 1 --2- 172.21.15.97:0/3829499063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd034004ac0 0x7fd034004ee0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd038009ac0 tx=0x7fd034006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d376a104d50594 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:34.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.030+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038039070 con 0x7fd034004ac0 2024-03-29T04:08:34.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.030+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd03802c6b0 con 0x7fd034004ac0 2024-03-29T04:08:34.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.030+0000 7fd0437fe640 1 -- 172.21.15.97:0/3829499063 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038009690 con 0x7fd034004ac0 2024-03-29T04:08:34.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.030+0000 7fd04ac9a640 1 -- 172.21.15.97:0/3829499063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd034004ac0 msgr2=0x7fd034004ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.030+0000 7fd04ac9a640 1 --2- 172.21.15.97:0/3829499063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd034004ac0 0x7fd034004ee0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd038009ac0 tx=0x7fd034006940 comp rx=0 tx=0).stop 2024-03-29T04:08:34.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.030+0000 7fd04ac9a640 1 -- 172.21.15.97:0/3829499063 shutdown_connections 2024-03-29T04:08:34.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.030+0000 7fd04ac9a640 1 --2- 172.21.15.97:0/3829499063 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd034004ac0 0x7fd034004ee0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.030+0000 7fd04ac9a640 1 -- 172.21.15.97:0/3829499063 >> 172.21.15.97:0/3829499063 conn(0x7fd04406b510 msgr2=0x7fd04406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:34.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.030+0000 7fd04ac9a640 1 -- 172.21.15.97:0/3829499063 shutdown_connections 2024-03-29T04:08:34.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.030+0000 7fd04ac9a640 1 -- 172.21.15.97:0/3829499063 wait complete. 2024-03-29T04:08:34.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.031+0000 7fd04ac9a640 1 Processor -- start 2024-03-29T04:08:34.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.031+0000 7fd04ac9a640 1 -- start start 2024-03-29T04:08:34.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.032+0000 7fd04ac9a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd034004ac0 0x7fd0440c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.032+0000 7fd04ac9a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0441f1760 0x7fd044071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.032+0000 7fd04ac9a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd044071ff0 0x7fd0440c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.032+0000 7fd04ac9a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd04406e650 con 0x7fd044071ff0 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.032+0000 7fd04ac9a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd0440b4790 con 0x7fd0441f1760 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.032+0000 7fd04ac9a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd04406e7d0 con 0x7fd034004ac0 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.032+0000 7fd049237640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd044071ff0 0x7fd0440c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.032+0000 7fd049237640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd044071ff0 0x7fd0440c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51254/0 (socket says 172.21.15.97:51254) 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.032+0000 7fd049237640 1 -- 172.21.15.97:0/545477365 learned_addr learned my addr 172.21.15.97:0/545477365 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.033+0000 7fd048a36640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd034004ac0 0x7fd0440c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.033+0000 7fd043fff640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0441f1760 0x7fd044071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.033+0000 7fd049237640 1 -- 172.21.15.97:0/545477365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd034004ac0 msgr2=0x7fd0440c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.033+0000 7fd049237640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd034004ac0 0x7fd0440c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.033+0000 7fd049237640 1 -- 172.21.15.97:0/545477365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0441f1760 msgr2=0x7fd044071ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.033+0000 7fd049237640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0441f1760 0x7fd044071ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.033+0000 7fd049237640 1 -- 172.21.15.97:0/545477365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04414bd80 con 0x7fd044071ff0 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.033+0000 7fd049237640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd044071ff0 0x7fd0440c92c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd044070e50 tx=0x7fd03c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:34.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.033+0000 7fd0417fa640 1 -- 172.21.15.97:0/545477365 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c003970 con 0x7fd044071ff0 2024-03-29T04:08:34.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.033+0000 7fd0417fa640 1 -- 172.21.15.97:0/545477365 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd03c018070 con 0x7fd044071ff0 2024-03-29T04:08:34.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.033+0000 7fd0417fa640 1 -- 172.21.15.97:0/545477365 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c003d50 con 0x7fd044071ff0 2024-03-29T04:08:34.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.033+0000 7fd04ac9a640 1 -- 172.21.15.97:0/545477365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04414c060 con 0x7fd044071ff0 2024-03-29T04:08:34.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.033+0000 7fd04ac9a640 1 -- 172.21.15.97:0/545477365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04414c620 con 0x7fd044071ff0 2024-03-29T04:08:34.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.034+0000 7fd04ac9a640 1 -- 172.21.15.97:0/545477365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd044068640 con 0x7fd044071ff0 2024-03-29T04:08:34.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.034+0000 7fd0417fa640 1 -- 172.21.15.97:0/545477365 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd03c0050d0 con 0x7fd044071ff0 2024-03-29T04:08:34.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.035+0000 7fd0417fa640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd02806c560 0x7fd02806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.035+0000 7fd048a36640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd02806c560 0x7fd02806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.035+0000 7fd0417fa640 1 -- 172.21.15.97:0/545477365 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd03c08af90 con 0x7fd044071ff0 2024-03-29T04:08:34.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.036+0000 7fd048a36640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd02806c560 0x7fd02806ea20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd038036000 tx=0x7fd038030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:34.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.039+0000 7fd0417fa640 1 -- 172.21.15.97:0/545477365 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd03c022020 con 0x7fd044071ff0 2024-03-29T04:08:34.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.323+0000 7fd04ac9a640 1 -- 172.21.15.97:0/545477365 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd04406c8b0 con 0x7fd02806c560 2024-03-29T04:08:34.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.328+0000 7fd0417fa640 1 -- 172.21.15.97:0/545477365 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237202 (secure 0 0 0) 0x7fd03807eeb0 con 0x7fd02806c560 2024-03-29T04:08:34.311 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:34.313 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-29T04:08:34.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.331+0000 7fd04ac9a640 1 -- 172.21.15.97:0/545477365 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd02806c560 msgr2=0x7fd02806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.331+0000 7fd04ac9a640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd02806c560 0x7fd02806ea20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd038036000 tx=0x7fd038030000 comp rx=0 tx=0).stop 2024-03-29T04:08:34.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.331+0000 7fd04ac9a640 1 -- 172.21.15.97:0/545477365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd044071ff0 msgr2=0x7fd0440c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.331+0000 7fd04ac9a640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd044071ff0 0x7fd0440c92c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd044070e50 tx=0x7fd03c00eb90 comp rx=0 tx=0).stop 2024-03-29T04:08:34.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.332+0000 7fd04ac9a640 1 -- 172.21.15.97:0/545477365 shutdown_connections 2024-03-29T04:08:34.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.332+0000 7fd04ac9a640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd02806c560 0x7fd02806ea20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.332+0000 7fd04ac9a640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd044071ff0 0x7fd0440c92c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.332+0000 7fd04ac9a640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0441f1760 0x7fd044071ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.332+0000 7fd04ac9a640 1 --2- 172.21.15.97:0/545477365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd034004ac0 0x7fd0440c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.332+0000 7fd04ac9a640 1 -- 172.21.15.97:0/545477365 >> 172.21.15.97:0/545477365 conn(0x7fd04406b510 msgr2=0x7fd0440aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:34.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.332+0000 7fd04ac9a640 1 -- 172.21.15.97:0/545477365 shutdown_connections 2024-03-29T04:08:34.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.332+0000 7fd04ac9a640 1 -- 172.21.15.97:0/545477365 wait complete. 2024-03-29T04:08:34.322 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-03-29T04:08:32.647549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219752,"kb_used_data":3520,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754968,"statfs":{"total":773094113280,"available":772869087232,"internally_reserved":0,"allocated":3604480,"data_stored":2073610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.050723"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:29.596344+0000","last_change":"2024-03-29T04:08:29.596344+0000","last_active":"2024-03-29T04:08:29.596344+0000","last_peered":"2024-03-29T04:08:29.596344+0000","last_clean":"2024-03-29T04:08:29.596344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:29.596344+0000","last_undegraded":"2024-03-29T04:08:29.596344+0000","last_fullsized":"2024-03-29T04:08:29.596344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T04:08:28.590894+0000","last_change":"2024-03-29T04:08:28.590894+0000","last_active":"2024-03-29T04:08:28.590894+0000","last_peered":"2024-03-29T04:08:28.590894+0000","last_clean":"2024-03-29T04:08:28.590894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:08:28.590894+0000","last_undegraded":"2024-03-29T04:08:28.590894+0000","last_fullsized":"2024-03-29T04:08:28.590894+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":184,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-29T04:08:24.603886+0000","last_change":"2024-03-29T04:07:29.856799+0000","last_active":"2024-03-29T04:08:24.603886+0000","last_peered":"2024-03-29T04:08:24.603886+0000","last_clean":"2024-03-29T04:08:24.603886+0000","last_became_active":"2024-03-29T04:07:29.855873+0000","last_became_peered":"2024-03-29T04:07:29.855873+0000","last_unstale":"2024-03-29T04:08:24.603886+0000","last_undegraded":"2024-03-29T04:08:24.603886+0000","last_fullsized":"2024-03-29T04:08:24.603886+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_clean_scrub_stamp":"2024-03-29T04:07:28.661516+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:25:15.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":905216,"data_stored":705770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27340,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":311296,"data_stored":115402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":105288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27336,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":311296,"data_stored":115402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27336,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":311296,"data_stored":115402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27332,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":311296,"data_stored":115402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":105288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":695656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:08:34.329 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:08:34.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.483+0000 7f2d11a55640 1 Processor -- start 2024-03-29T04:08:34.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.483+0000 7f2d11a55640 1 -- start start 2024-03-29T04:08:34.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.484+0000 7f2d11a55640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2d0c0723a0 0x7f2d0c0af820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.484+0000 7f2d11a55640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2d0c0cdb80 0x7f2d0c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.484+0000 7f2d11a55640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d0c0b0210 con 0x7f2d0c15ab90 2024-03-29T04:08:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.484+0000 7f2d11a55640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2d0c0cdfd0 con 0x7f2d0c0723a0 2024-03-29T04:08:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.484+0000 7f2d11a55640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f2d0c06dcf0 con 0x7f2d0c0cdb80 2024-03-29T04:08:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.484+0000 7f2d03fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2d0c0723a0 0x7f2d0c0af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.484+0000 7f2d03fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2d0c0723a0 0x7f2d0c0af820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51262/0 (socket says 172.21.15.97:51262) 2024-03-29T04:08:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.485+0000 7f2d03fff640 1 -- 172.21.15.97:0/3739193164 learned_addr learned my addr 172.21.15.97:0/3739193164 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.485+0000 7f2d0b7fe640 1 --2- 172.21.15.97:0/3739193164 >> v2:172.21.15.154:3300/0 conn(0x7f2d0c0cdb80 0x7f2d0c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.485+0000 7f2d03fff640 1 -- 172.21.15.97:0/3739193164 >> v2:172.21.15.154:3300/0 conn(0x7f2d0c0cdb80 msgr2=0x7f2d0c0b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.485+0000 7f2d03fff640 1 --2- 172.21.15.97:0/3739193164 >> v2:172.21.15.154:3300/0 conn(0x7f2d0c0cdb80 0x7f2d0c0b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.485+0000 7f2d03fff640 1 -- 172.21.15.97:0/3739193164 >> v1:172.21.15.154:6789/0 conn(0x7f2d0c15ab90 legacy=0x7f2d0c071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.485+0000 7f2d03fff640 1 -- 172.21.15.97:0/3739193164 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c06df40 con 0x7f2d0c0723a0 2024-03-29T04:08:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.486+0000 7f2d03fff640 1 --2- 172.21.15.97:0/3739193164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d0c0723a0 0x7f2d0c0af820 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0afd60 tx=0x7f2cf402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26685f3b02e762c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.486+0000 7f2d0a7fc640 1 -- 172.21.15.97:0/3739193164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4035020 con 0x7f2d0c0723a0 2024-03-29T04:08:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.486+0000 7f2d0a7fc640 1 -- 172.21.15.97:0/3739193164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf4031870 con 0x7f2d0c0723a0 2024-03-29T04:08:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.486+0000 7f2d0a7fc640 1 -- 172.21.15.97:0/3739193164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4031c40 con 0x7f2d0c0723a0 2024-03-29T04:08:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.486+0000 7f2d11a55640 1 -- 172.21.15.97:0/3739193164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d0c0723a0 msgr2=0x7f2d0c0af820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.486+0000 7f2d11a55640 1 --2- 172.21.15.97:0/3739193164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d0c0723a0 0x7f2d0c0af820 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0afd60 tx=0x7f2cf402b910 comp rx=0 tx=0).stop 2024-03-29T04:08:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.486+0000 7f2d11a55640 1 -- 172.21.15.97:0/3739193164 shutdown_connections 2024-03-29T04:08:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.486+0000 7f2d11a55640 1 --2- 172.21.15.97:0/3739193164 >> v2:172.21.15.154:3300/0 conn(0x7f2d0c0cdb80 0x7f2d0c0b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.486+0000 7f2d11a55640 1 --2- 172.21.15.97:0/3739193164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d0c0723a0 0x7f2d0c0af820 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.486+0000 7f2d11a55640 1 -- 172.21.15.97:0/3739193164 >> 172.21.15.97:0/3739193164 conn(0x7f2d0c0d0730 msgr2=0x7f2d0c0d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.486+0000 7f2d11a55640 1 -- 172.21.15.97:0/3739193164 shutdown_connections 2024-03-29T04:08:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.486+0000 7f2d11a55640 1 -- 172.21.15.97:0/3739193164 wait complete. 2024-03-29T04:08:34.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.487+0000 7f2d11a55640 1 Processor -- start 2024-03-29T04:08:34.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.488+0000 7f2d11a55640 1 -- start start 2024-03-29T04:08:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.488+0000 7f2d11a55640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d0c0723a0 0x7f2d0c155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.488+0000 7f2d11a55640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d0c0cdb80 0x7f2d0c14f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.488+0000 7f2d11a55640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d0c15ab90 0x7f2d0c148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.488+0000 7f2d11a55640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c15af70 con 0x7f2d0c15ab90 2024-03-29T04:08:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.488+0000 7f2d11a55640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c06e500 con 0x7f2d0c0723a0 2024-03-29T04:08:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.488+0000 7f2d11a55640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0c15b0f0 con 0x7f2d0c0cdb80 2024-03-29T04:08:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.488+0000 7f2d0b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d0c15ab90 0x7f2d0c148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.488+0000 7f2d0b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d0c15ab90 0x7f2d0c148ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51278/0 (socket says 172.21.15.97:51278) 2024-03-29T04:08:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.488+0000 7f2d0b7fe640 1 -- 172.21.15.97:0/262051370 learned_addr learned my addr 172.21.15.97:0/262051370 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.489+0000 7f2d0b7fe640 1 -- 172.21.15.97:0/262051370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d0c0cdb80 msgr2=0x7f2d0c14f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.489+0000 7f2d0b7fe640 1 --2- 172.21.15.97:0/262051370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d0c0cdb80 0x7f2d0c14f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.489+0000 7f2d0b7fe640 1 -- 172.21.15.97:0/262051370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d0c0723a0 msgr2=0x7f2d0c155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.489+0000 7f2d0b7fe640 1 --2- 172.21.15.97:0/262051370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d0c0723a0 0x7f2d0c155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.489+0000 7f2d0b7fe640 1 -- 172.21.15.97:0/262051370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c155550 con 0x7f2d0c15ab90 2024-03-29T04:08:34.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.489+0000 7f2d0b7fe640 1 --2- 172.21.15.97:0/262051370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d0c15ab90 0x7f2d0c148ab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2d04007e40 tx=0x7f2d0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.489+0000 7f2d08ff9640 1 -- 172.21.15.97:0/262051370 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04014620 con 0x7f2d0c15ab90 2024-03-29T04:08:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.489+0000 7f2d08ff9640 1 -- 172.21.15.97:0/262051370 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d04019070 con 0x7f2d0c15ab90 2024-03-29T04:08:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.489+0000 7f2d11a55640 1 -- 172.21.15.97:0/262051370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0c154f90 con 0x7f2d0c15ab90 2024-03-29T04:08:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.489+0000 7f2d08ff9640 1 -- 172.21.15.97:0/262051370 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04014a20 con 0x7f2d0c15ab90 2024-03-29T04:08:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.489+0000 7f2d11a55640 1 -- 172.21.15.97:0/262051370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0c15b310 con 0x7f2d0c15ab90 2024-03-29T04:08:34.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.490+0000 7f2d11a55640 1 -- 172.21.15.97:0/262051370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d0c15b640 con 0x7f2d0c15ab90 2024-03-29T04:08:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.490+0000 7f2d08ff9640 1 -- 172.21.15.97:0/262051370 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2d04014bc0 con 0x7f2d0c15ab90 2024-03-29T04:08:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.491+0000 7f2d08ff9640 1 --2- 172.21.15.97:0/262051370 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2cec06c530 0x7f2cec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.491+0000 7f2d08ff9640 1 -- 172.21.15.97:0/262051370 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2d0408bff0 con 0x7f2d0c15ab90 2024-03-29T04:08:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.491+0000 7f2d0affd640 1 --2- 172.21.15.97:0/262051370 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2cec06c530 0x7f2cec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.491+0000 7f2d0affd640 1 --2- 172.21.15.97:0/262051370 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2cec06c530 0x7f2cec06e9f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc000f60 tx=0x7f2cfc0079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.495+0000 7f2d08ff9640 1 -- 172.21.15.97:0/262051370 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2d0405a610 con 0x7f2d0c15ab90 2024-03-29T04:08:34.563 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-03-29T04:08:34.583+0000 7f7a4710f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T04:08:34.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.772+0000 7f2d11a55640 1 -- 172.21.15.97:0/262051370 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d0c0cca80 con 0x7f2cec06c530 2024-03-29T04:08:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.778+0000 7f2d08ff9640 1 -- 172.21.15.97:0/262051370 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251922 (secure 0 0 0) 0x7f2d0c0cca80 con 0x7f2cec06c530 2024-03-29T04:08:34.760 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:34.762 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-29T04:08:34.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.782+0000 7f2d11a55640 1 -- 172.21.15.97:0/262051370 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2cec06c530 msgr2=0x7f2cec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.782+0000 7f2d11a55640 1 --2- 172.21.15.97:0/262051370 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2cec06c530 0x7f2cec06e9f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc000f60 tx=0x7f2cfc0079d0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.782+0000 7f2d11a55640 1 -- 172.21.15.97:0/262051370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d0c15ab90 msgr2=0x7f2d0c148ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.782+0000 7f2d11a55640 1 --2- 172.21.15.97:0/262051370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d0c15ab90 0x7f2d0c148ab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2d04007e40 tx=0x7f2d0400e4b0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.782+0000 7f2d11a55640 1 -- 172.21.15.97:0/262051370 shutdown_connections 2024-03-29T04:08:34.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.782+0000 7f2d11a55640 1 --2- 172.21.15.97:0/262051370 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2cec06c530 0x7f2cec06e9f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.782+0000 7f2d11a55640 1 --2- 172.21.15.97:0/262051370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d0c15ab90 0x7f2d0c148ab0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.782+0000 7f2d11a55640 1 --2- 172.21.15.97:0/262051370 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d0c0cdb80 0x7f2d0c14f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.782+0000 7f2d11a55640 1 --2- 172.21.15.97:0/262051370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d0c0723a0 0x7f2d0c155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.782+0000 7f2d11a55640 1 -- 172.21.15.97:0/262051370 >> 172.21.15.97:0/262051370 conn(0x7f2d0c0d0730 msgr2=0x7f2d0c0b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:34.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.782+0000 7f2d11a55640 1 -- 172.21.15.97:0/262051370 shutdown_connections 2024-03-29T04:08:34.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.782+0000 7f2d11a55640 1 -- 172.21.15.97:0/262051370 wait complete. 2024-03-29T04:08:34.773 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-03-29T04:08:34.647834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220200,"kb_used_data":3992,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754520,"statfs":{"total":773094113280,"available":772868628480,"internally_reserved":0,"allocated":4087808,"data_stored":2284844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000873"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.628004+0000","last_change":"2024-03-29T04:08:30.626795+0000","last_active":"2024-03-29T04:08:31.628004+0000","last_peered":"2024-03-29T04:08:31.628004+0000","last_clean":"2024-03-29T04:08:31.628004+0000","last_became_active":"2024-03-29T04:08:30.626515+0000","last_became_peered":"2024-03-29T04:08:30.626515+0000","last_unstale":"2024-03-29T04:08:31.628004+0000","last_undegraded":"2024-03-29T04:08:31.628004+0000","last_fullsized":"2024-03-29T04:08:31.628004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:41:30.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626548+0000","last_change":"2024-03-29T04:08:30.617376+0000","last_active":"2024-03-29T04:08:31.626548+0000","last_peered":"2024-03-29T04:08:31.626548+0000","last_clean":"2024-03-29T04:08:31.626548+0000","last_became_active":"2024-03-29T04:08:30.617095+0000","last_became_peered":"2024-03-29T04:08:30.617095+0000","last_unstale":"2024-03-29T04:08:31.626548+0000","last_undegraded":"2024-03-29T04:08:31.626548+0000","last_fullsized":"2024-03-29T04:08:31.626548+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:41:38.704966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627294+0000","last_change":"2024-03-29T04:08:30.632868+0000","last_active":"2024-03-29T04:08:31.627294+0000","last_peered":"2024-03-29T04:08:31.627294+0000","last_clean":"2024-03-29T04:08:31.627294+0000","last_became_active":"2024-03-29T04:08:30.632360+0000","last_became_peered":"2024-03-29T04:08:30.632360+0000","last_unstale":"2024-03-29T04:08:31.627294+0000","last_undegraded":"2024-03-29T04:08:31.627294+0000","last_fullsized":"2024-03-29T04:08:31.627294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:08.298094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.636876+0000","last_change":"2024-03-29T04:08:30.627789+0000","last_active":"2024-03-29T04:08:30.636876+0000","last_peered":"2024-03-29T04:08:30.636876+0000","last_clean":"2024-03-29T04:08:30.636876+0000","last_became_active":"2024-03-29T04:08:30.626991+0000","last_became_peered":"2024-03-29T04:08:30.626991+0000","last_unstale":"2024-03-29T04:08:30.636876+0000","last_undegraded":"2024-03-29T04:08:30.636876+0000","last_fullsized":"2024-03-29T04:08:30.636876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:41:30.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:32.319309+0000","last_change":"2024-03-29T04:08:30.624634+0000","last_active":"2024-03-29T04:08:32.319309+0000","last_peered":"2024-03-29T04:08:32.319309+0000","last_clean":"2024-03-29T04:08:32.319309+0000","last_became_active":"2024-03-29T04:08:30.624334+0000","last_became_peered":"2024-03-29T04:08:30.624334+0000","last_unstale":"2024-03-29T04:08:32.319309+0000","last_undegraded":"2024-03-29T04:08:32.319309+0000","last_fullsized":"2024-03-29T04:08:32.319309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:12:01.436987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626288+0000","last_change":"2024-03-29T04:08:30.631353+0000","last_active":"2024-03-29T04:08:31.626288+0000","last_peered":"2024-03-29T04:08:31.626288+0000","last_clean":"2024-03-29T04:08:31.626288+0000","last_became_active":"2024-03-29T04:08:30.631058+0000","last_became_peered":"2024-03-29T04:08:30.631058+0000","last_unstale":"2024-03-29T04:08:31.626288+0000","last_undegraded":"2024-03-29T04:08:31.626288+0000","last_fullsized":"2024-03-29T04:08:31.626288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:43:27.306071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.634482+0000","last_change":"2024-03-29T04:08:30.627320+0000","last_active":"2024-03-29T04:08:30.634482+0000","last_peered":"2024-03-29T04:08:30.634482+0000","last_clean":"2024-03-29T04:08:30.634482+0000","last_became_active":"2024-03-29T04:08:30.627032+0000","last_became_peered":"2024-03-29T04:08:30.627032+0000","last_unstale":"2024-03-29T04:08:30.634482+0000","last_undegraded":"2024-03-29T04:08:30.634482+0000","last_fullsized":"2024-03-29T04:08:30.634482+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:02.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.625699+0000","last_change":"2024-03-29T04:08:30.620138+0000","last_active":"2024-03-29T04:08:30.625699+0000","last_peered":"2024-03-29T04:08:30.625699+0000","last_clean":"2024-03-29T04:08:30.625699+0000","last_became_active":"2024-03-29T04:08:30.619825+0000","last_became_peered":"2024-03-29T04:08:30.619825+0000","last_unstale":"2024-03-29T04:08:30.625699+0000","last_undegraded":"2024-03-29T04:08:30.625699+0000","last_fullsized":"2024-03-29T04:08:30.625699+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:09.838288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878568+0000","last_change":"2024-03-29T04:08:30.629144+0000","last_active":"2024-03-29T04:08:31.878568+0000","last_peered":"2024-03-29T04:08:31.878568+0000","last_clean":"2024-03-29T04:08:31.878568+0000","last_became_active":"2024-03-29T04:08:30.628752+0000","last_became_peered":"2024-03-29T04:08:30.628752+0000","last_unstale":"2024-03-29T04:08:31.878568+0000","last_undegraded":"2024-03-29T04:08:31.878568+0000","last_fullsized":"2024-03-29T04:08:31.878568+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:31.641060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.636863+0000","last_change":"2024-03-29T04:08:30.629672+0000","last_active":"2024-03-29T04:08:30.636863+0000","last_peered":"2024-03-29T04:08:30.636863+0000","last_clean":"2024-03-29T04:08:30.636863+0000","last_became_active":"2024-03-29T04:08:30.629109+0000","last_became_peered":"2024-03-29T04:08:30.629109+0000","last_unstale":"2024-03-29T04:08:30.636863+0000","last_undegraded":"2024-03-29T04:08:30.636863+0000","last_fullsized":"2024-03-29T04:08:30.636863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:11:07.151605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.626632+0000","last_change":"2024-03-29T04:08:30.621835+0000","last_active":"2024-03-29T04:08:30.626632+0000","last_peered":"2024-03-29T04:08:30.626632+0000","last_clean":"2024-03-29T04:08:30.626632+0000","last_became_active":"2024-03-29T04:08:30.621551+0000","last_became_peered":"2024-03-29T04:08:30.621551+0000","last_unstale":"2024-03-29T04:08:30.626632+0000","last_undegraded":"2024-03-29T04:08:30.626632+0000","last_fullsized":"2024-03-29T04:08:30.626632+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:09.717633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627010+0000","last_change":"2024-03-29T04:08:30.634201+0000","last_active":"2024-03-29T04:08:31.627010+0000","last_peered":"2024-03-29T04:08:31.627010+0000","last_clean":"2024-03-29T04:08:31.627010+0000","last_became_active":"2024-03-29T04:08:30.633792+0000","last_became_peered":"2024-03-29T04:08:30.633792+0000","last_unstale":"2024-03-29T04:08:31.627010+0000","last_undegraded":"2024-03-29T04:08:31.627010+0000","last_fullsized":"2024-03-29T04:08:31.627010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:11:03.855488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:32.319321+0000","last_change":"2024-03-29T04:08:30.619203+0000","last_active":"2024-03-29T04:08:32.319321+0000","last_peered":"2024-03-29T04:08:32.319321+0000","last_clean":"2024-03-29T04:08:32.319321+0000","last_became_active":"2024-03-29T04:08:30.618908+0000","last_became_peered":"2024-03-29T04:08:30.618908+0000","last_unstale":"2024-03-29T04:08:32.319321+0000","last_undegraded":"2024-03-29T04:08:32.319321+0000","last_fullsized":"2024-03-29T04:08:32.319321+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:19:04.007129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626272+0000","last_change":"2024-03-29T04:08:30.622827+0000","last_active":"2024-03-29T04:08:31.626272+0000","last_peered":"2024-03-29T04:08:31.626272+0000","last_clean":"2024-03-29T04:08:31.626272+0000","last_became_active":"2024-03-29T04:08:30.620491+0000","last_became_peered":"2024-03-29T04:08:30.620491+0000","last_unstale":"2024-03-29T04:08:31.626272+0000","last_undegraded":"2024-03-29T04:08:31.626272+0000","last_fullsized":"2024-03-29T04:08:31.626272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:14.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627090+0000","last_change":"2024-03-29T04:08:30.625908+0000","last_active":"2024-03-29T04:08:31.627090+0000","last_peered":"2024-03-29T04:08:31.627090+0000","last_clean":"2024-03-29T04:08:31.627090+0000","last_became_active":"2024-03-29T04:08:30.625628+0000","last_became_peered":"2024-03-29T04:08:30.625628+0000","last_unstale":"2024-03-29T04:08:31.627090+0000","last_undegraded":"2024-03-29T04:08:31.627090+0000","last_fullsized":"2024-03-29T04:08:31.627090+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:14.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.628057+0000","last_change":"2024-03-29T04:08:30.624280+0000","last_active":"2024-03-29T04:08:30.628057+0000","last_peered":"2024-03-29T04:08:30.628057+0000","last_clean":"2024-03-29T04:08:30.628057+0000","last_became_active":"2024-03-29T04:08:30.623974+0000","last_became_peered":"2024-03-29T04:08:30.623974+0000","last_unstale":"2024-03-29T04:08:30.628057+0000","last_undegraded":"2024-03-29T04:08:30.628057+0000","last_fullsized":"2024-03-29T04:08:30.628057+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:10:08.272599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.634203+0000","last_change":"2024-03-29T04:08:30.627233+0000","last_active":"2024-03-29T04:08:30.634203+0000","last_peered":"2024-03-29T04:08:30.634203+0000","last_clean":"2024-03-29T04:08:30.634203+0000","last_became_active":"2024-03-29T04:08:30.626968+0000","last_became_peered":"2024-03-29T04:08:30.626968+0000","last_unstale":"2024-03-29T04:08:30.634203+0000","last_undegraded":"2024-03-29T04:08:30.634203+0000","last_fullsized":"2024-03-29T04:08:30.634203+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:31.641060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626579+0000","last_change":"2024-03-29T04:08:30.626820+0000","last_active":"2024-03-29T04:08:31.626579+0000","last_peered":"2024-03-29T04:08:31.626579+0000","last_clean":"2024-03-29T04:08:31.626579+0000","last_became_active":"2024-03-29T04:08:30.626170+0000","last_became_peered":"2024-03-29T04:08:30.626170+0000","last_unstale":"2024-03-29T04:08:31.626579+0000","last_undegraded":"2024-03-29T04:08:31.626579+0000","last_fullsized":"2024-03-29T04:08:31.626579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:36:43.631557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626358+0000","last_change":"2024-03-29T04:08:30.620202+0000","last_active":"2024-03-29T04:08:31.626358+0000","last_peered":"2024-03-29T04:08:31.626358+0000","last_clean":"2024-03-29T04:08:31.626358+0000","last_became_active":"2024-03-29T04:08:30.619915+0000","last_became_peered":"2024-03-29T04:08:30.619915+0000","last_unstale":"2024-03-29T04:08:31.626358+0000","last_undegraded":"2024-03-29T04:08:31.626358+0000","last_fullsized":"2024-03-29T04:08:31.626358+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:08.298094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:32.319469+0000","last_change":"2024-03-29T04:08:30.631073+0000","last_active":"2024-03-29T04:08:32.319469+0000","last_peered":"2024-03-29T04:08:32.319469+0000","last_clean":"2024-03-29T04:08:32.319469+0000","last_became_active":"2024-03-29T04:08:30.630794+0000","last_became_peered":"2024-03-29T04:08:30.630794+0000","last_unstale":"2024-03-29T04:08:32.319469+0000","last_undegraded":"2024-03-29T04:08:32.319469+0000","last_fullsized":"2024-03-29T04:08:32.319469+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:10.440113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878034+0000","last_change":"2024-03-29T04:08:30.626521+0000","last_active":"2024-03-29T04:08:31.878034+0000","last_peered":"2024-03-29T04:08:31.878034+0000","last_clean":"2024-03-29T04:08:31.878034+0000","last_became_active":"2024-03-29T04:08:30.626266+0000","last_became_peered":"2024-03-29T04:08:30.626266+0000","last_unstale":"2024-03-29T04:08:31.878034+0000","last_undegraded":"2024-03-29T04:08:31.878034+0000","last_fullsized":"2024-03-29T04:08:31.878034+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:09.717633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878076+0000","last_change":"2024-03-29T04:08:30.631398+0000","last_active":"2024-03-29T04:08:31.878076+0000","last_peered":"2024-03-29T04:08:31.878076+0000","last_clean":"2024-03-29T04:08:31.878076+0000","last_became_active":"2024-03-29T04:08:30.631080+0000","last_became_peered":"2024-03-29T04:08:30.631080+0000","last_unstale":"2024-03-29T04:08:31.878076+0000","last_undegraded":"2024-03-29T04:08:31.878076+0000","last_fullsized":"2024-03-29T04:08:31.878076+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:10:08.272599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.621441+0000","last_change":"2024-03-29T04:08:30.615228+0000","last_active":"2024-03-29T04:08:30.621441+0000","last_peered":"2024-03-29T04:08:30.621441+0000","last_clean":"2024-03-29T04:08:30.621441+0000","last_became_active":"2024-03-29T04:08:30.614885+0000","last_became_peered":"2024-03-29T04:08:30.614885+0000","last_unstale":"2024-03-29T04:08:30.621441+0000","last_undegraded":"2024-03-29T04:08:30.621441+0000","last_fullsized":"2024-03-29T04:08:30.621441+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:17:08.003574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626743+0000","last_change":"2024-03-29T04:08:30.625026+0000","last_active":"2024-03-29T04:08:31.626743+0000","last_peered":"2024-03-29T04:08:31.626743+0000","last_clean":"2024-03-29T04:08:31.626743+0000","last_became_active":"2024-03-29T04:08:30.624724+0000","last_became_peered":"2024-03-29T04:08:30.624724+0000","last_unstale":"2024-03-29T04:08:31.626743+0000","last_undegraded":"2024-03-29T04:08:31.626743+0000","last_fullsized":"2024-03-29T04:08:31.626743+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:49:36.774393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.631607+0000","last_change":"2024-03-29T04:08:30.623626+0000","last_active":"2024-03-29T04:08:30.631607+0000","last_peered":"2024-03-29T04:08:30.631607+0000","last_clean":"2024-03-29T04:08:30.631607+0000","last_became_active":"2024-03-29T04:08:30.623224+0000","last_became_peered":"2024-03-29T04:08:30.623224+0000","last_unstale":"2024-03-29T04:08:30.631607+0000","last_undegraded":"2024-03-29T04:08:30.631607+0000","last_fullsized":"2024-03-29T04:08:30.631607+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:29:39.403419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626620+0000","last_change":"2024-03-29T04:08:30.626193+0000","last_active":"2024-03-29T04:08:31.626620+0000","last_peered":"2024-03-29T04:08:31.626620+0000","last_clean":"2024-03-29T04:08:31.626620+0000","last_became_active":"2024-03-29T04:08:30.625921+0000","last_became_peered":"2024-03-29T04:08:30.625921+0000","last_unstale":"2024-03-29T04:08:31.626620+0000","last_undegraded":"2024-03-29T04:08:31.626620+0000","last_fullsized":"2024-03-29T04:08:31.626620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:14.535406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.636542+0000","last_change":"2024-03-29T04:08:30.626140+0000","last_active":"2024-03-29T04:08:30.636542+0000","last_peered":"2024-03-29T04:08:30.636542+0000","last_clean":"2024-03-29T04:08:30.636542+0000","last_became_active":"2024-03-29T04:08:30.625891+0000","last_became_peered":"2024-03-29T04:08:30.625891+0000","last_unstale":"2024-03-29T04:08:30.636542+0000","last_undegraded":"2024-03-29T04:08:30.636542+0000","last_fullsized":"2024-03-29T04:08:30.636542+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:05:02.707650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627070+0000","last_change":"2024-03-29T04:08:30.625583+0000","last_active":"2024-03-29T04:08:31.627070+0000","last_peered":"2024-03-29T04:08:31.627070+0000","last_clean":"2024-03-29T04:08:31.627070+0000","last_became_active":"2024-03-29T04:08:30.625076+0000","last_became_peered":"2024-03-29T04:08:30.625076+0000","last_unstale":"2024-03-29T04:08:31.627070+0000","last_undegraded":"2024-03-29T04:08:31.627070+0000","last_fullsized":"2024-03-29T04:08:31.627070+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:56:36.457470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878069+0000","last_change":"2024-03-29T04:08:30.631406+0000","last_active":"2024-03-29T04:08:31.878069+0000","last_peered":"2024-03-29T04:08:31.878069+0000","last_clean":"2024-03-29T04:08:31.878069+0000","last_became_active":"2024-03-29T04:08:30.631113+0000","last_became_peered":"2024-03-29T04:08:30.631113+0000","last_unstale":"2024-03-29T04:08:31.878069+0000","last_undegraded":"2024-03-29T04:08:31.878069+0000","last_fullsized":"2024-03-29T04:08:31.878069+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:41:30.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.625732+0000","last_change":"2024-03-29T04:08:30.621317+0000","last_active":"2024-03-29T04:08:30.625732+0000","last_peered":"2024-03-29T04:08:30.625732+0000","last_clean":"2024-03-29T04:08:30.625732+0000","last_became_active":"2024-03-29T04:08:30.621025+0000","last_became_peered":"2024-03-29T04:08:30.621025+0000","last_unstale":"2024-03-29T04:08:30.625732+0000","last_undegraded":"2024-03-29T04:08:30.625732+0000","last_fullsized":"2024-03-29T04:08:30.625732+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:45:31.756271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627933+0000","last_change":"2024-03-29T04:08:30.622034+0000","last_active":"2024-03-29T04:08:31.627933+0000","last_peered":"2024-03-29T04:08:31.627933+0000","last_clean":"2024-03-29T04:08:31.627933+0000","last_became_active":"2024-03-29T04:08:30.621756+0000","last_became_peered":"2024-03-29T04:08:30.621756+0000","last_unstale":"2024-03-29T04:08:31.627933+0000","last_undegraded":"2024-03-29T04:08:31.627933+0000","last_fullsized":"2024-03-29T04:08:31.627933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:05:02.707650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.629233+0000","last_change":"2024-03-29T04:08:30.625792+0000","last_active":"2024-03-29T04:08:30.629233+0000","last_peered":"2024-03-29T04:08:30.629233+0000","last_clean":"2024-03-29T04:08:30.629233+0000","last_became_active":"2024-03-29T04:08:30.625429+0000","last_became_peered":"2024-03-29T04:08:30.625429+0000","last_unstale":"2024-03-29T04:08:30.629233+0000","last_undegraded":"2024-03-29T04:08:30.629233+0000","last_fullsized":"2024-03-29T04:08:30.629233+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:44:48.614263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.622423+0000","last_change":"2024-03-29T04:08:30.615736+0000","last_active":"2024-03-29T04:08:30.622423+0000","last_peered":"2024-03-29T04:08:30.622423+0000","last_clean":"2024-03-29T04:08:30.622423+0000","last_became_active":"2024-03-29T04:08:30.614800+0000","last_became_peered":"2024-03-29T04:08:30.614800+0000","last_unstale":"2024-03-29T04:08:30.622423+0000","last_undegraded":"2024-03-29T04:08:30.622423+0000","last_fullsized":"2024-03-29T04:08:30.622423+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:29:39.403419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.628091+0000","last_change":"2024-03-29T04:08:29.618250+0000","last_active":"2024-03-29T04:08:31.628091+0000","last_peered":"2024-03-29T04:08:31.628091+0000","last_clean":"2024-03-29T04:08:31.628091+0000","last_became_active":"2024-03-29T04:08:29.617971+0000","last_became_peered":"2024-03-29T04:08:29.617971+0000","last_unstale":"2024-03-29T04:08:31.628091+0000","last_undegraded":"2024-03-29T04:08:31.628091+0000","last_fullsized":"2024-03-29T04:08:31.628091+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:39:49.963357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.628041+0000","last_change":"2024-03-29T04:08:30.629764+0000","last_active":"2024-03-29T04:08:31.628041+0000","last_peered":"2024-03-29T04:08:31.628041+0000","last_clean":"2024-03-29T04:08:31.628041+0000","last_became_active":"2024-03-29T04:08:30.629415+0000","last_became_peered":"2024-03-29T04:08:30.629415+0000","last_unstale":"2024-03-29T04:08:31.628041+0000","last_undegraded":"2024-03-29T04:08:31.628041+0000","last_fullsized":"2024-03-29T04:08:31.628041+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:10:08.272599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.625215+0000","last_change":"2024-03-29T04:08:29.626329+0000","last_active":"2024-03-29T04:08:30.625215+0000","last_peered":"2024-03-29T04:08:30.625215+0000","last_clean":"2024-03-29T04:08:30.625215+0000","last_became_active":"2024-03-29T04:08:29.626108+0000","last_became_peered":"2024-03-29T04:08:29.626108+0000","last_unstale":"2024-03-29T04:08:30.625215+0000","last_undegraded":"2024-03-29T04:08:30.625215+0000","last_fullsized":"2024-03-29T04:08:30.625215+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:03:00.552561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878282+0000","last_change":"2024-03-29T04:08:30.634175+0000","last_active":"2024-03-29T04:08:31.878282+0000","last_peered":"2024-03-29T04:08:31.878282+0000","last_clean":"2024-03-29T04:08:31.878282+0000","last_became_active":"2024-03-29T04:08:30.633878+0000","last_became_peered":"2024-03-29T04:08:30.633878+0000","last_unstale":"2024-03-29T04:08:31.878282+0000","last_undegraded":"2024-03-29T04:08:31.878282+0000","last_fullsized":"2024-03-29T04:08:31.878282+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:11:07.151605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626409+0000","last_change":"2024-03-29T04:08:29.623534+0000","last_active":"2024-03-29T04:08:31.626409+0000","last_peered":"2024-03-29T04:08:31.626409+0000","last_clean":"2024-03-29T04:08:31.626409+0000","last_became_active":"2024-03-29T04:08:29.623261+0000","last_became_peered":"2024-03-29T04:08:29.623261+0000","last_unstale":"2024-03-29T04:08:31.626409+0000","last_undegraded":"2024-03-29T04:08:31.626409+0000","last_fullsized":"2024-03-29T04:08:31.626409+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:05:01.702200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.636516+0000","last_change":"2024-03-29T04:08:30.625484+0000","last_active":"2024-03-29T04:08:30.636516+0000","last_peered":"2024-03-29T04:08:30.636516+0000","last_clean":"2024-03-29T04:08:30.636516+0000","last_became_active":"2024-03-29T04:08:30.625153+0000","last_became_peered":"2024-03-29T04:08:30.625153+0000","last_unstale":"2024-03-29T04:08:30.636516+0000","last_undegraded":"2024-03-29T04:08:30.636516+0000","last_fullsized":"2024-03-29T04:08:30.636516+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:09.838288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:32.319408+0000","last_change":"2024-03-29T04:08:29.622457+0000","last_active":"2024-03-29T04:08:32.319408+0000","last_peered":"2024-03-29T04:08:32.319408+0000","last_clean":"2024-03-29T04:08:32.319408+0000","last_became_active":"2024-03-29T04:08:29.622191+0000","last_became_peered":"2024-03-29T04:08:29.622191+0000","last_unstale":"2024-03-29T04:08:32.319408+0000","last_undegraded":"2024-03-29T04:08:32.319408+0000","last_fullsized":"2024-03-29T04:08:32.319408+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:53:51.959355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.628012+0000","last_change":"2024-03-29T04:08:30.625031+0000","last_active":"2024-03-29T04:08:30.628012+0000","last_peered":"2024-03-29T04:08:30.628012+0000","last_clean":"2024-03-29T04:08:30.628012+0000","last_became_active":"2024-03-29T04:08:30.624729+0000","last_became_peered":"2024-03-29T04:08:30.624729+0000","last_unstale":"2024-03-29T04:08:30.628012+0000","last_undegraded":"2024-03-29T04:08:30.628012+0000","last_fullsized":"2024-03-29T04:08:30.628012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:11:07.151605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878189+0000","last_change":"2024-03-29T04:08:29.619377+0000","last_active":"2024-03-29T04:08:31.878189+0000","last_peered":"2024-03-29T04:08:31.878189+0000","last_clean":"2024-03-29T04:08:31.878189+0000","last_became_active":"2024-03-29T04:08:29.619083+0000","last_became_peered":"2024-03-29T04:08:29.619083+0000","last_unstale":"2024-03-29T04:08:31.878189+0000","last_undegraded":"2024-03-29T04:08:31.878189+0000","last_fullsized":"2024-03-29T04:08:31.878189+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:24:02.866540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627013+0000","last_change":"2024-03-29T04:08:30.627727+0000","last_active":"2024-03-29T04:08:31.627013+0000","last_peered":"2024-03-29T04:08:31.627013+0000","last_clean":"2024-03-29T04:08:31.627013+0000","last_became_active":"2024-03-29T04:08:30.627375+0000","last_became_peered":"2024-03-29T04:08:30.627375+0000","last_unstale":"2024-03-29T04:08:31.627013+0000","last_undegraded":"2024-03-29T04:08:31.627013+0000","last_fullsized":"2024-03-29T04:08:31.627013+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:58.175312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.877970+0000","last_change":"2024-03-29T04:08:29.614504+0000","last_active":"2024-03-29T04:08:31.877970+0000","last_peered":"2024-03-29T04:08:31.877970+0000","last_clean":"2024-03-29T04:08:31.877970+0000","last_became_active":"2024-03-29T04:08:29.614226+0000","last_became_peered":"2024-03-29T04:08:29.614226+0000","last_unstale":"2024-03-29T04:08:31.877970+0000","last_undegraded":"2024-03-29T04:08:31.877970+0000","last_fullsized":"2024-03-29T04:08:31.877970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:46:28.211351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878030+0000","last_change":"2024-03-29T04:08:30.623129+0000","last_active":"2024-03-29T04:08:31.878030+0000","last_peered":"2024-03-29T04:08:31.878030+0000","last_clean":"2024-03-29T04:08:31.878030+0000","last_became_active":"2024-03-29T04:08:30.622852+0000","last_became_peered":"2024-03-29T04:08:30.622852+0000","last_unstale":"2024-03-29T04:08:31.878030+0000","last_undegraded":"2024-03-29T04:08:31.878030+0000","last_fullsized":"2024-03-29T04:08:31.878030+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:45:31.756271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626083+0000","last_change":"2024-03-29T04:08:29.618110+0000","last_active":"2024-03-29T04:08:31.626083+0000","last_peered":"2024-03-29T04:08:31.626083+0000","last_clean":"2024-03-29T04:08:31.626083+0000","last_became_active":"2024-03-29T04:08:29.617831+0000","last_became_peered":"2024-03-29T04:08:29.617831+0000","last_unstale":"2024-03-29T04:08:31.626083+0000","last_undegraded":"2024-03-29T04:08:31.626083+0000","last_fullsized":"2024-03-29T04:08:31.626083+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:08.832839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.636635+0000","last_change":"2024-03-29T04:08:30.627603+0000","last_active":"2024-03-29T04:08:30.636635+0000","last_peered":"2024-03-29T04:08:30.636635+0000","last_clean":"2024-03-29T04:08:30.636635+0000","last_became_active":"2024-03-29T04:08:30.627339+0000","last_became_peered":"2024-03-29T04:08:30.627339+0000","last_unstale":"2024-03-29T04:08:30.636635+0000","last_undegraded":"2024-03-29T04:08:30.636635+0000","last_fullsized":"2024-03-29T04:08:30.636635+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:09.717633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.877931+0000","last_change":"2024-03-29T04:08:29.615838+0000","last_active":"2024-03-29T04:08:31.877931+0000","last_peered":"2024-03-29T04:08:31.877931+0000","last_clean":"2024-03-29T04:08:31.877931+0000","last_became_active":"2024-03-29T04:08:29.615560+0000","last_became_peered":"2024-03-29T04:08:29.615560+0000","last_unstale":"2024-03-29T04:08:31.877931+0000","last_undegraded":"2024-03-29T04:08:31.877931+0000","last_fullsized":"2024-03-29T04:08:31.877931+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:28.273971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627355+0000","last_change":"2024-03-29T04:08:30.632532+0000","last_active":"2024-03-29T04:08:31.627355+0000","last_peered":"2024-03-29T04:08:31.627355+0000","last_clean":"2024-03-29T04:08:31.627355+0000","last_became_active":"2024-03-29T04:08:30.632228+0000","last_became_peered":"2024-03-29T04:08:30.632228+0000","last_unstale":"2024-03-29T04:08:31.627355+0000","last_undegraded":"2024-03-29T04:08:31.627355+0000","last_fullsized":"2024-03-29T04:08:31.627355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:53:21.090141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626429+0000","last_change":"2024-03-29T04:08:29.615005+0000","last_active":"2024-03-29T04:08:31.626429+0000","last_peered":"2024-03-29T04:08:31.626429+0000","last_clean":"2024-03-29T04:08:31.626429+0000","last_became_active":"2024-03-29T04:08:29.614609+0000","last_became_peered":"2024-03-29T04:08:29.614609+0000","last_unstale":"2024-03-29T04:08:31.626429+0000","last_undegraded":"2024-03-29T04:08:31.626429+0000","last_fullsized":"2024-03-29T04:08:31.626429+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:08.712183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626436+0000","last_change":"2024-03-29T04:08:30.622669+0000","last_active":"2024-03-29T04:08:31.626436+0000","last_peered":"2024-03-29T04:08:31.626436+0000","last_clean":"2024-03-29T04:08:31.626436+0000","last_became_active":"2024-03-29T04:08:30.622398+0000","last_became_peered":"2024-03-29T04:08:30.622398+0000","last_unstale":"2024-03-29T04:08:31.626436+0000","last_undegraded":"2024-03-29T04:08:31.626436+0000","last_fullsized":"2024-03-29T04:08:31.626436+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:53:21.090141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.605535+0000","last_change":"2024-03-29T04:08:29.624176+0000","last_active":"2024-03-29T04:08:30.605535+0000","last_peered":"2024-03-29T04:08:30.605535+0000","last_clean":"2024-03-29T04:08:30.605535+0000","last_became_active":"2024-03-29T04:08:29.623912+0000","last_became_peered":"2024-03-29T04:08:29.623912+0000","last_unstale":"2024-03-29T04:08:30.605535+0000","last_undegraded":"2024-03-29T04:08:30.605535+0000","last_fullsized":"2024-03-29T04:08:30.605535+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:39:49.963357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626456+0000","last_change":"2024-03-29T04:08:30.625442+0000","last_active":"2024-03-29T04:08:31.626456+0000","last_peered":"2024-03-29T04:08:31.626456+0000","last_clean":"2024-03-29T04:08:31.626456+0000","last_became_active":"2024-03-29T04:08:30.625146+0000","last_became_peered":"2024-03-29T04:08:30.625146+0000","last_unstale":"2024-03-29T04:08:31.626456+0000","last_undegraded":"2024-03-29T04:08:31.626456+0000","last_fullsized":"2024-03-29T04:08:31.626456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:54:01.716836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627384+0000","last_change":"2024-03-29T04:08:29.617062+0000","last_active":"2024-03-29T04:08:31.627384+0000","last_peered":"2024-03-29T04:08:31.627384+0000","last_clean":"2024-03-29T04:08:31.627384+0000","last_became_active":"2024-03-29T04:08:29.616698+0000","last_became_peered":"2024-03-29T04:08:29.616698+0000","last_unstale":"2024-03-29T04:08:31.627384+0000","last_undegraded":"2024-03-29T04:08:31.627384+0000","last_fullsized":"2024-03-29T04:08:31.627384+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:05:01.702200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627217+0000","last_change":"2024-03-29T04:08:30.632860+0000","last_active":"2024-03-29T04:08:31.627217+0000","last_peered":"2024-03-29T04:08:31.627217+0000","last_clean":"2024-03-29T04:08:31.627217+0000","last_became_active":"2024-03-29T04:08:30.632445+0000","last_became_peered":"2024-03-29T04:08:30.632445+0000","last_unstale":"2024-03-29T04:08:31.627217+0000","last_undegraded":"2024-03-29T04:08:31.627217+0000","last_fullsized":"2024-03-29T04:08:31.627217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:43:27.306071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626448+0000","last_change":"2024-03-29T04:08:29.622557+0000","last_active":"2024-03-29T04:08:31.626448+0000","last_peered":"2024-03-29T04:08:31.626448+0000","last_clean":"2024-03-29T04:08:31.626448+0000","last_became_active":"2024-03-29T04:08:29.622302+0000","last_became_peered":"2024-03-29T04:08:29.622302+0000","last_unstale":"2024-03-29T04:08:31.626448+0000","last_undegraded":"2024-03-29T04:08:31.626448+0000","last_fullsized":"2024-03-29T04:08:31.626448+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:41:29.424210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.637033+0000","last_change":"2024-03-29T04:08:30.629502+0000","last_active":"2024-03-29T04:08:30.637033+0000","last_peered":"2024-03-29T04:08:30.637033+0000","last_clean":"2024-03-29T04:08:30.637033+0000","last_became_active":"2024-03-29T04:08:30.629246+0000","last_became_peered":"2024-03-29T04:08:30.629246+0000","last_unstale":"2024-03-29T04:08:30.637033+0000","last_undegraded":"2024-03-29T04:08:30.637033+0000","last_fullsized":"2024-03-29T04:08:30.637033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:10:08.272599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878204+0000","last_change":"2024-03-29T04:08:29.613354+0000","last_active":"2024-03-29T04:08:31.878204+0000","last_peered":"2024-03-29T04:08:31.878204+0000","last_clean":"2024-03-29T04:08:31.878204+0000","last_became_active":"2024-03-29T04:08:29.612903+0000","last_became_peered":"2024-03-29T04:08:29.612903+0000","last_unstale":"2024-03-29T04:08:31.878204+0000","last_undegraded":"2024-03-29T04:08:31.878204+0000","last_fullsized":"2024-03-29T04:08:31.878204+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:13:34.988897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626458+0000","last_change":"2024-03-29T04:08:30.629447+0000","last_active":"2024-03-29T04:08:31.626458+0000","last_peered":"2024-03-29T04:08:31.626458+0000","last_clean":"2024-03-29T04:08:31.626458+0000","last_became_active":"2024-03-29T04:08:30.629180+0000","last_became_peered":"2024-03-29T04:08:30.629180+0000","last_unstale":"2024-03-29T04:08:31.626458+0000","last_undegraded":"2024-03-29T04:08:31.626458+0000","last_fullsized":"2024-03-29T04:08:31.626458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:15:42.573740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.610566+0000","last_change":"2024-03-29T04:08:29.621563+0000","last_active":"2024-03-29T04:08:30.610566+0000","last_peered":"2024-03-29T04:08:30.610566+0000","last_clean":"2024-03-29T04:08:30.610566+0000","last_became_active":"2024-03-29T04:08:29.621220+0000","last_became_peered":"2024-03-29T04:08:29.621220+0000","last_unstale":"2024-03-29T04:08:30.610566+0000","last_undegraded":"2024-03-29T04:08:30.610566+0000","last_fullsized":"2024-03-29T04:08:30.610566+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:48:56.190185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.628038+0000","last_change":"2024-03-29T04:08:30.622551+0000","last_active":"2024-03-29T04:08:30.628038+0000","last_peered":"2024-03-29T04:08:30.628038+0000","last_clean":"2024-03-29T04:08:30.628038+0000","last_became_active":"2024-03-29T04:08:30.622266+0000","last_became_peered":"2024-03-29T04:08:30.622266+0000","last_unstale":"2024-03-29T04:08:30.628038+0000","last_undegraded":"2024-03-29T04:08:30.628038+0000","last_fullsized":"2024-03-29T04:08:30.628038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:41:30.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626148+0000","last_change":"2024-03-29T04:08:29.614228+0000","last_active":"2024-03-29T04:08:31.626148+0000","last_peered":"2024-03-29T04:08:31.626148+0000","last_clean":"2024-03-29T04:08:31.626148+0000","last_became_active":"2024-03-29T04:08:29.612505+0000","last_became_peered":"2024-03-29T04:08:29.612505+0000","last_unstale":"2024-03-29T04:08:31.626148+0000","last_undegraded":"2024-03-29T04:08:31.626148+0000","last_fullsized":"2024-03-29T04:08:31.626148+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:45:30.750821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627933+0000","last_change":"2024-03-29T04:08:30.630395+0000","last_active":"2024-03-29T04:08:31.627933+0000","last_peered":"2024-03-29T04:08:31.627933+0000","last_clean":"2024-03-29T04:08:31.627933+0000","last_became_active":"2024-03-29T04:08:30.629992+0000","last_became_peered":"2024-03-29T04:08:30.629992+0000","last_unstale":"2024-03-29T04:08:31.627933+0000","last_undegraded":"2024-03-29T04:08:31.627933+0000","last_fullsized":"2024-03-29T04:08:31.627933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:45:31.756271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626111+0000","last_change":"2024-03-29T04:08:29.621902+0000","last_active":"2024-03-29T04:08:31.626111+0000","last_peered":"2024-03-29T04:08:31.626111+0000","last_clean":"2024-03-29T04:08:31.626111+0000","last_became_active":"2024-03-29T04:08:29.621654+0000","last_became_peered":"2024-03-29T04:08:29.621654+0000","last_unstale":"2024-03-29T04:08:31.626111+0000","last_undegraded":"2024-03-29T04:08:31.626111+0000","last_fullsized":"2024-03-29T04:08:31.626111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:17:06.998124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878458+0000","last_change":"2024-03-29T04:08:30.627334+0000","last_active":"2024-03-29T04:08:31.878458+0000","last_peered":"2024-03-29T04:08:31.878458+0000","last_clean":"2024-03-29T04:08:31.878458+0000","last_became_active":"2024-03-29T04:08:30.627003+0000","last_became_peered":"2024-03-29T04:08:30.627003+0000","last_unstale":"2024-03-29T04:08:31.878458+0000","last_undegraded":"2024-03-29T04:08:31.878458+0000","last_fullsized":"2024-03-29T04:08:31.878458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:44:48.614263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.633683+0000","last_change":"2024-03-29T04:08:29.618510+0000","last_active":"2024-03-29T04:08:30.633683+0000","last_peered":"2024-03-29T04:08:30.633683+0000","last_clean":"2024-03-29T04:08:30.633683+0000","last_became_active":"2024-03-29T04:08:29.618235+0000","last_became_peered":"2024-03-29T04:08:29.618235+0000","last_unstale":"2024-03-29T04:08:30.633683+0000","last_undegraded":"2024-03-29T04:08:30.633683+0000","last_fullsized":"2024-03-29T04:08:30.633683+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:09.434663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627451+0000","last_change":"2024-03-29T04:08:30.626770+0000","last_active":"2024-03-29T04:08:31.627451+0000","last_peered":"2024-03-29T04:08:31.627451+0000","last_clean":"2024-03-29T04:08:31.627451+0000","last_became_active":"2024-03-29T04:08:30.626512+0000","last_became_peered":"2024-03-29T04:08:30.626512+0000","last_unstale":"2024-03-29T04:08:31.627451+0000","last_undegraded":"2024-03-29T04:08:31.627451+0000","last_fullsized":"2024-03-29T04:08:31.627451+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:54:01.716836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878407+0000","last_change":"2024-03-29T04:08:29.617171+0000","last_active":"2024-03-29T04:08:31.878407+0000","last_peered":"2024-03-29T04:08:31.878407+0000","last_clean":"2024-03-29T04:08:31.878407+0000","last_became_active":"2024-03-29T04:08:29.616823+0000","last_became_peered":"2024-03-29T04:08:29.616823+0000","last_unstale":"2024-03-29T04:08:31.878407+0000","last_undegraded":"2024-03-29T04:08:31.878407+0000","last_fullsized":"2024-03-29T04:08:31.878407+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:45:39.427545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.636836+0000","last_change":"2024-03-29T04:08:30.627532+0000","last_active":"2024-03-29T04:08:30.636836+0000","last_peered":"2024-03-29T04:08:30.636836+0000","last_clean":"2024-03-29T04:08:30.636836+0000","last_became_active":"2024-03-29T04:08:30.627221+0000","last_became_peered":"2024-03-29T04:08:30.627221+0000","last_unstale":"2024-03-29T04:08:30.636836+0000","last_undegraded":"2024-03-29T04:08:30.636836+0000","last_fullsized":"2024-03-29T04:08:30.636836+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:45:31.756271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878432+0000","last_change":"2024-03-29T04:08:29.617323+0000","last_active":"2024-03-29T04:08:31.878432+0000","last_peered":"2024-03-29T04:08:31.878432+0000","last_clean":"2024-03-29T04:08:31.878432+0000","last_became_active":"2024-03-29T04:08:29.616466+0000","last_became_peered":"2024-03-29T04:08:29.616466+0000","last_unstale":"2024-03-29T04:08:31.878432+0000","last_undegraded":"2024-03-29T04:08:31.878432+0000","last_fullsized":"2024-03-29T04:08:31.878432+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:19:03.001679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:32.319357+0000","last_change":"2024-03-29T04:08:30.623985+0000","last_active":"2024-03-29T04:08:32.319357+0000","last_peered":"2024-03-29T04:08:32.319357+0000","last_clean":"2024-03-29T04:08:32.319357+0000","last_became_active":"2024-03-29T04:08:30.623736+0000","last_became_peered":"2024-03-29T04:08:30.623736+0000","last_unstale":"2024-03-29T04:08:32.319357+0000","last_undegraded":"2024-03-29T04:08:32.319357+0000","last_fullsized":"2024-03-29T04:08:32.319357+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:24:03.871990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878334+0000","last_change":"2024-03-29T04:08:29.615349+0000","last_active":"2024-03-29T04:08:31.878334+0000","last_peered":"2024-03-29T04:08:31.878334+0000","last_clean":"2024-03-29T04:08:31.878334+0000","last_became_active":"2024-03-29T04:08:29.614611+0000","last_became_peered":"2024-03-29T04:08:29.614611+0000","last_unstale":"2024-03-29T04:08:31.878334+0000","last_undegraded":"2024-03-29T04:08:31.878334+0000","last_fullsized":"2024-03-29T04:08:31.878334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:56:46.205739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:32.319384+0000","last_change":"2024-03-29T04:08:30.625808+0000","last_active":"2024-03-29T04:08:32.319384+0000","last_peered":"2024-03-29T04:08:32.319384+0000","last_clean":"2024-03-29T04:08:32.319384+0000","last_became_active":"2024-03-29T04:08:30.625537+0000","last_became_peered":"2024-03-29T04:08:30.625537+0000","last_unstale":"2024-03-29T04:08:32.319384+0000","last_undegraded":"2024-03-29T04:08:32.319384+0000","last_fullsized":"2024-03-29T04:08:32.319384+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:56:47.211189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.636734+0000","last_change":"2024-03-29T04:08:29.625276+0000","last_active":"2024-03-29T04:08:30.636734+0000","last_peered":"2024-03-29T04:08:30.636734+0000","last_clean":"2024-03-29T04:08:30.636734+0000","last_became_active":"2024-03-29T04:08:29.625011+0000","last_became_peered":"2024-03-29T04:08:29.625011+0000","last_unstale":"2024-03-29T04:08:30.636734+0000","last_undegraded":"2024-03-29T04:08:30.636734+0000","last_fullsized":"2024-03-29T04:08:30.636734+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:09.434663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:32.319452+0000","last_change":"2024-03-29T04:08:30.626820+0000","last_active":"2024-03-29T04:08:32.319452+0000","last_peered":"2024-03-29T04:08:32.319452+0000","last_clean":"2024-03-29T04:08:32.319452+0000","last_became_active":"2024-03-29T04:08:30.626543+0000","last_became_peered":"2024-03-29T04:08:30.626543+0000","last_unstale":"2024-03-29T04:08:32.319452+0000","last_undegraded":"2024-03-29T04:08:32.319452+0000","last_fullsized":"2024-03-29T04:08:32.319452+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:03:01.558010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627854+0000","last_change":"2024-03-29T04:08:29.621302+0000","last_active":"2024-03-29T04:08:31.627854+0000","last_peered":"2024-03-29T04:08:31.627854+0000","last_clean":"2024-03-29T04:08:31.627854+0000","last_became_active":"2024-03-29T04:08:29.621076+0000","last_became_peered":"2024-03-29T04:08:29.621076+0000","last_unstale":"2024-03-29T04:08:31.627854+0000","last_undegraded":"2024-03-29T04:08:31.627854+0000","last_fullsized":"2024-03-29T04:08:31.627854+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:09.434663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:32.319310+0000","last_change":"2024-03-29T04:08:30.623310+0000","last_active":"2024-03-29T04:08:32.319310+0000","last_peered":"2024-03-29T04:08:32.319310+0000","last_clean":"2024-03-29T04:08:32.319310+0000","last_became_active":"2024-03-29T04:08:30.623042+0000","last_became_peered":"2024-03-29T04:08:30.623042+0000","last_unstale":"2024-03-29T04:08:32.319310+0000","last_undegraded":"2024-03-29T04:08:32.319310+0000","last_fullsized":"2024-03-29T04:08:32.319310+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:13:35.994347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878204+0000","last_change":"2024-03-29T04:08:29.611504+0000","last_active":"2024-03-29T04:08:31.878204+0000","last_peered":"2024-03-29T04:08:31.878204+0000","last_clean":"2024-03-29T04:08:31.878204+0000","last_became_active":"2024-03-29T04:08:29.611020+0000","last_became_peered":"2024-03-29T04:08:29.611020+0000","last_unstale":"2024-03-29T04:08:31.878204+0000","last_undegraded":"2024-03-29T04:08:31.878204+0000","last_fullsized":"2024-03-29T04:08:31.878204+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:12:57.721653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.634504+0000","last_change":"2024-03-29T04:08:30.628033+0000","last_active":"2024-03-29T04:08:30.634504+0000","last_peered":"2024-03-29T04:08:30.634504+0000","last_clean":"2024-03-29T04:08:30.634504+0000","last_became_active":"2024-03-29T04:08:30.627735+0000","last_became_peered":"2024-03-29T04:08:30.627735+0000","last_unstale":"2024-03-29T04:08:30.634504+0000","last_undegraded":"2024-03-29T04:08:30.634504+0000","last_fullsized":"2024-03-29T04:08:30.634504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:05:54.082160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626680+0000","last_change":"2024-03-29T04:08:29.613387+0000","last_active":"2024-03-29T04:08:31.626680+0000","last_peered":"2024-03-29T04:08:31.626680+0000","last_clean":"2024-03-29T04:08:31.626680+0000","last_became_active":"2024-03-29T04:08:29.613083+0000","last_became_peered":"2024-03-29T04:08:29.613083+0000","last_unstale":"2024-03-29T04:08:31.626680+0000","last_undegraded":"2024-03-29T04:08:31.626680+0000","last_fullsized":"2024-03-29T04:08:31.626680+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:08.832839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:32.319424+0000","last_change":"2024-03-29T04:08:30.630211+0000","last_active":"2024-03-29T04:08:32.319424+0000","last_peered":"2024-03-29T04:08:32.319424+0000","last_clean":"2024-03-29T04:08:32.319424+0000","last_became_active":"2024-03-29T04:08:30.629943+0000","last_became_peered":"2024-03-29T04:08:30.629943+0000","last_unstale":"2024-03-29T04:08:32.319424+0000","last_undegraded":"2024-03-29T04:08:32.319424+0000","last_fullsized":"2024-03-29T04:08:32.319424+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:11.511045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.631559+0000","last_change":"2024-03-29T04:08:29.616795+0000","last_active":"2024-03-29T04:08:30.631559+0000","last_peered":"2024-03-29T04:08:30.631559+0000","last_clean":"2024-03-29T04:08:30.631559+0000","last_became_active":"2024-03-29T04:08:29.616514+0000","last_became_peered":"2024-03-29T04:08:29.616514+0000","last_unstale":"2024-03-29T04:08:30.631559+0000","last_undegraded":"2024-03-29T04:08:30.631559+0000","last_fullsized":"2024-03-29T04:08:30.631559+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:39:49.963357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627514+0000","last_change":"2024-03-29T04:08:30.632688+0000","last_active":"2024-03-29T04:08:31.627514+0000","last_peered":"2024-03-29T04:08:31.627514+0000","last_clean":"2024-03-29T04:08:31.627514+0000","last_became_active":"2024-03-29T04:08:30.632353+0000","last_became_peered":"2024-03-29T04:08:30.632353+0000","last_unstale":"2024-03-29T04:08:31.627514+0000","last_undegraded":"2024-03-29T04:08:31.627514+0000","last_fullsized":"2024-03-29T04:08:31.627514+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:15:42.573740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"31'1","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.608263+0000","last_change":"2024-03-29T04:08:29.625319+0000","last_active":"2024-03-29T04:08:31.608263+0000","last_peered":"2024-03-29T04:08:31.608263+0000","last_clean":"2024-03-29T04:08:31.608263+0000","last_became_active":"2024-03-29T04:08:29.624964+0000","last_became_peered":"2024-03-29T04:08:29.624964+0000","last_unstale":"2024-03-29T04:08:31.608263+0000","last_undegraded":"2024-03-29T04:08:31.608263+0000","last_fullsized":"2024-03-29T04:08:31.608263+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:48:56.190185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.624192+0000","last_change":"2024-03-29T04:08:30.616634+0000","last_active":"2024-03-29T04:08:30.624192+0000","last_peered":"2024-03-29T04:08:30.624192+0000","last_clean":"2024-03-29T04:08:30.624192+0000","last_became_active":"2024-03-29T04:08:30.616301+0000","last_became_peered":"2024-03-29T04:08:30.616301+0000","last_unstale":"2024-03-29T04:08:30.624192+0000","last_undegraded":"2024-03-29T04:08:30.624192+0000","last_fullsized":"2024-03-29T04:08:30.624192+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:09.838288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.609166+0000","last_change":"2024-03-29T04:08:29.617554+0000","last_active":"2024-03-29T04:08:30.609166+0000","last_peered":"2024-03-29T04:08:30.609166+0000","last_clean":"2024-03-29T04:08:30.609166+0000","last_became_active":"2024-03-29T04:08:29.617311+0000","last_became_peered":"2024-03-29T04:08:29.617311+0000","last_unstale":"2024-03-29T04:08:30.609166+0000","last_undegraded":"2024-03-29T04:08:30.609166+0000","last_fullsized":"2024-03-29T04:08:30.609166+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:06:19.657549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627982+0000","last_change":"2024-03-29T04:08:30.622744+0000","last_active":"2024-03-29T04:08:31.627982+0000","last_peered":"2024-03-29T04:08:31.627982+0000","last_clean":"2024-03-29T04:08:31.627982+0000","last_became_active":"2024-03-29T04:08:30.622425+0000","last_became_peered":"2024-03-29T04:08:30.622425+0000","last_unstale":"2024-03-29T04:08:31.627982+0000","last_undegraded":"2024-03-29T04:08:31.627982+0000","last_fullsized":"2024-03-29T04:08:31.627982+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:09.717633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.605612+0000","last_change":"2024-03-29T04:08:29.626354+0000","last_active":"2024-03-29T04:08:30.605612+0000","last_peered":"2024-03-29T04:08:30.605612+0000","last_clean":"2024-03-29T04:08:30.605612+0000","last_became_active":"2024-03-29T04:08:29.626102+0000","last_became_peered":"2024-03-29T04:08:29.626102+0000","last_unstale":"2024-03-29T04:08:30.605612+0000","last_undegraded":"2024-03-29T04:08:30.605612+0000","last_fullsized":"2024-03-29T04:08:30.605612+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:06:19.657549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.625304+0000","last_change":"2024-03-29T04:08:30.620783+0000","last_active":"2024-03-29T04:08:30.625304+0000","last_peered":"2024-03-29T04:08:30.625304+0000","last_clean":"2024-03-29T04:08:30.625304+0000","last_became_active":"2024-03-29T04:08:30.620218+0000","last_became_peered":"2024-03-29T04:08:30.620218+0000","last_unstale":"2024-03-29T04:08:30.625304+0000","last_undegraded":"2024-03-29T04:08:30.625304+0000","last_fullsized":"2024-03-29T04:08:30.625304+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:05:02.707650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.610566+0000","last_change":"2024-03-29T04:08:29.616771+0000","last_active":"2024-03-29T04:08:30.610566+0000","last_peered":"2024-03-29T04:08:30.610566+0000","last_clean":"2024-03-29T04:08:30.610566+0000","last_became_active":"2024-03-29T04:08:29.616500+0000","last_became_peered":"2024-03-29T04:08:29.616500+0000","last_unstale":"2024-03-29T04:08:30.610566+0000","last_undegraded":"2024-03-29T04:08:30.610566+0000","last_fullsized":"2024-03-29T04:08:30.610566+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:17:06.998124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626076+0000","last_change":"2024-03-29T04:08:30.617616+0000","last_active":"2024-03-29T04:08:31.626076+0000","last_peered":"2024-03-29T04:08:31.626076+0000","last_clean":"2024-03-29T04:08:31.626076+0000","last_became_active":"2024-03-29T04:08:30.617364+0000","last_became_peered":"2024-03-29T04:08:30.617364+0000","last_unstale":"2024-03-29T04:08:31.626076+0000","last_undegraded":"2024-03-29T04:08:31.626076+0000","last_fullsized":"2024-03-29T04:08:31.626076+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:56:36.457470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.878158+0000","last_change":"2024-03-29T04:08:29.614347+0000","last_active":"2024-03-29T04:08:31.878158+0000","last_peered":"2024-03-29T04:08:31.878158+0000","last_clean":"2024-03-29T04:08:31.878158+0000","last_became_active":"2024-03-29T04:08:29.613986+0000","last_became_peered":"2024-03-29T04:08:29.613986+0000","last_unstale":"2024-03-29T04:08:31.878158+0000","last_undegraded":"2024-03-29T04:08:31.878158+0000","last_fullsized":"2024-03-29T04:08:31.878158+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:12:00.431538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.877936+0000","last_change":"2024-03-29T04:08:30.625382+0000","last_active":"2024-03-29T04:08:31.877936+0000","last_peered":"2024-03-29T04:08:31.877936+0000","last_clean":"2024-03-29T04:08:31.877936+0000","last_became_active":"2024-03-29T04:08:30.624876+0000","last_became_peered":"2024-03-29T04:08:30.624876+0000","last_unstale":"2024-03-29T04:08:31.877936+0000","last_undegraded":"2024-03-29T04:08:31.877936+0000","last_fullsized":"2024-03-29T04:08:31.877936+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:05:02.707650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.626171+0000","last_change":"2024-03-29T04:08:29.612944+0000","last_active":"2024-03-29T04:08:31.626171+0000","last_peered":"2024-03-29T04:08:31.626171+0000","last_clean":"2024-03-29T04:08:31.626171+0000","last_became_active":"2024-03-29T04:08:29.612665+0000","last_became_peered":"2024-03-29T04:08:29.612665+0000","last_unstale":"2024-03-29T04:08:31.626171+0000","last_undegraded":"2024-03-29T04:08:31.626171+0000","last_fullsized":"2024-03-29T04:08:31.626171+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:29:38.397970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.625771+0000","last_change":"2024-03-29T04:08:30.620139+0000","last_active":"2024-03-29T04:08:30.625771+0000","last_peered":"2024-03-29T04:08:30.625771+0000","last_clean":"2024-03-29T04:08:30.625771+0000","last_became_active":"2024-03-29T04:08:30.619825+0000","last_became_peered":"2024-03-29T04:08:30.619825+0000","last_unstale":"2024-03-29T04:08:30.625771+0000","last_undegraded":"2024-03-29T04:08:30.625771+0000","last_fullsized":"2024-03-29T04:08:30.625771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:29.596344+0000","last_clean_scrub_stamp":"2024-03-29T04:08:29.596344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:05:02.707650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:31.627959+0000","last_change":"2024-03-29T04:08:29.617445+0000","last_active":"2024-03-29T04:08:31.627959+0000","last_peered":"2024-03-29T04:08:31.627959+0000","last_clean":"2024-03-29T04:08:31.627959+0000","last_became_active":"2024-03-29T04:08:29.617193+0000","last_became_peered":"2024-03-29T04:08:29.617193+0000","last_unstale":"2024-03-29T04:08:31.627959+0000","last_undegraded":"2024-03-29T04:08:31.627959+0000","last_fullsized":"2024-03-29T04:08:31.627959+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:08:28.590894+0000","last_clean_scrub_stamp":"2024-03-29T04:08:28.590894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:03:00.552561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-29T04:08:30.610324+0000","last_change":"2024-03-29T04:07:29.856799+0000","last_active":"2024-03-29T04:08:30.610324+0000","last_peered":"2024-03-29T04:08:30.610324+0000","last_clean":"2024-03-29T04:08:30.610324+0000","last_became_active":"2024-03-29T04:07:29.855873+0000","last_became_peered":"2024-03-29T04:07:29.855873+0000","last_unstale":"2024-03-29T04:08:30.610324+0000","last_undegraded":"2024-03-29T04:08:30.610324+0000","last_fullsized":"2024-03-29T04:08:30.610324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_clean_scrub_stamp":"2024-03-29T04:07:28.661516+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:25:15.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":946176,"data_stored":724719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":368640,"data_stored":141177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":368640,"data_stored":141233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":368640,"data_stored":140665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":142195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":134331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":135895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":724629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:08:34.777 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T04:08:34.777 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-29T04:08:34.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.942+0000 7f40a3b60640 1 Processor -- start 2024-03-29T04:08:34.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.943+0000 7f40a3b60640 1 -- start start 2024-03-29T04:08:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.944+0000 7f40a3b60640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f409c09aa30 0x7f409c15f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.944+0000 7f40a3b60640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f409c0b3f30 con 0x7f409c06b510 2024-03-29T04:08:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.944+0000 7f40a3b60640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f409c0b3c50 con 0x7f409c0ae320 2024-03-29T04:08:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.944+0000 7f40a3b60640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f409c0b3a00 con 0x7f409c09aa30 2024-03-29T04:08:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.944+0000 7f40a18fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f409c06b510 0x7f409c06b8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55810/0 (socket says 172.21.15.97:55810) 2024-03-29T04:08:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.944+0000 7f40a18fc640 1 -- 172.21.15.97:0/1319339654 learned_addr learned my addr 172.21.15.97:0/1319339654 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.944+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3002962565 0 0) 0x7f409c0b3f30 con 0x7f409c06b510 2024-03-29T04:08:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.944+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4088003520 con 0x7f409c06b510 2024-03-29T04:08:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2882745161 0 0) 0x7f4088003520 con 0x7f409c06b510 2024-03-29T04:08:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 >> v2:172.21.15.154:3300/0 conn(0x7f409c09aa30 msgr2=0x7f409c15f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a08fa640 1 --2- 172.21.15.97:0/1319339654 >> v2:172.21.15.154:3300/0 conn(0x7f409c09aa30 0x7f409c15f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 >> v1:172.21.15.154:6789/0 conn(0x7f409c0ae320 legacy=0x7f409c15e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c15dea0 con 0x7f409c06b510 2024-03-29T04:08:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f408c001670 con 0x7f409c06b510 2024-03-29T04:08:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 >> v1:172.21.15.97:6789/0 conn(0x7f409c06b510 legacy=0x7f409c06b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a08fa640 1 --2- 172.21.15.97:0/1319339654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40880039b0 0x7f4088003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a08fa640 1 --2- 172.21.15.97:0/1319339654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4088004800 0x7f4088004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a08fa640 1 --2- 172.21.15.97:0/1319339654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40880052e0 0x7f408800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f409c0b3a00 con 0x7f40880039b0 2024-03-29T04:08:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f408800c1e0 con 0x7f40880052e0 2024-03-29T04:08:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f408800c3e0 con 0x7f4088004800 2024-03-29T04:08:34.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a18fc640 1 --2- 172.21.15.97:0/1319339654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40880039b0 0x7f4088003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a20fd640 1 --2- 172.21.15.97:0/1319339654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40880052e0 0x7f408800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a10fb640 1 --2- 172.21.15.97:0/1319339654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4088004800 0x7f4088004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a18fc640 1 -- 172.21.15.97:0/1319339654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4088004800 msgr2=0x7f4088004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:34.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a18fc640 1 --2- 172.21.15.97:0/1319339654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4088004800 0x7f4088004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a18fc640 1 -- 172.21.15.97:0/1319339654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40880052e0 msgr2=0x7f408800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a18fc640 1 --2- 172.21.15.97:0/1319339654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40880052e0 0x7f408800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a18fc640 1 -- 172.21.15.97:0/1319339654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4088003520 con 0x7f40880039b0 2024-03-29T04:08:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.945+0000 7f40a20fd640 1 -- 172.21.15.97:0/1319339654 reap_dead start 2024-03-29T04:08:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.946+0000 7f40a18fc640 1 --2- 172.21.15.97:0/1319339654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40880039b0 0x7f4088003dd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c210 tx=0x7f4088005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7d97a294e7a5a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.946+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c007d30 con 0x7f40880039b0 2024-03-29T04:08:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.946+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f408c030070 con 0x7f40880039b0 2024-03-29T04:08:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.946+0000 7f40a08fa640 1 -- 172.21.15.97:0/1319339654 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c02cda0 con 0x7f40880039b0 2024-03-29T04:08:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.947+0000 7f40a3b60640 1 -- 172.21.15.97:0/1319339654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40880039b0 msgr2=0x7f4088003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.947+0000 7f40a3b60640 1 --2- 172.21.15.97:0/1319339654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40880039b0 0x7f4088003dd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c210 tx=0x7f4088005830 comp rx=0 tx=0).stop 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.947+0000 7f40a3b60640 1 -- 172.21.15.97:0/1319339654 shutdown_connections 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.947+0000 7f40a3b60640 1 --2- 172.21.15.97:0/1319339654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40880039b0 0x7f4088003dd0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.947+0000 7f40a3b60640 1 -- 172.21.15.97:0/1319339654 >> 172.21.15.97:0/1319339654 conn(0x7f409c072e40 msgr2=0x7f409c0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.947+0000 7f40a3b60640 1 -- 172.21.15.97:0/1319339654 shutdown_connections 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.947+0000 7f40a3b60640 1 -- 172.21.15.97:0/1319339654 wait complete. 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.948+0000 7f40a3b60640 1 Processor -- start 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.948+0000 7f40a3b60640 1 -- start start 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a3b60640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40880039b0 0x7f409c13b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a3b60640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f409c154670 0x7f409c13e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a3b60640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f409c13eae0 0x7f409c1f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a3b60640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f409c0b67c0 con 0x7f409c154670 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a3b60640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f409c15e4f0 con 0x7f40880039b0 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a3b60640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f409c145660 con 0x7f409c13eae0 2024-03-29T04:08:34.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a10fb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f409c154670 0x7f409c13e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a10fb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f409c154670 0x7f409c13e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51298/0 (socket says 172.21.15.97:51298) 2024-03-29T04:08:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a10fb640 1 -- 172.21.15.97:0/72359717 learned_addr learned my addr 172.21.15.97:0/72359717 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a18fc640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40880039b0 0x7f409c13b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a20fd640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f409c13eae0 0x7f409c1f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a10fb640 1 -- 172.21.15.97:0/72359717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f409c13eae0 msgr2=0x7f409c1f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a10fb640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f409c13eae0 0x7f409c1f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a10fb640 1 -- 172.21.15.97:0/72359717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40880039b0 msgr2=0x7f409c13b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a10fb640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40880039b0 0x7f409c13b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a10fb640 1 -- 172.21.15.97:0/72359717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c13d520 con 0x7f409c154670 2024-03-29T04:08:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a20fd640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f409c13eae0 0x7f409c1f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a10fb640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f409c154670 0x7f409c13e5a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f409800f820 tx=0x7f409800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:34.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40927fc640 1 -- 172.21.15.97:0/72359717 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40980050d0 con 0x7f409c154670 2024-03-29T04:08:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40927fc640 1 -- 172.21.15.97:0/72359717 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4098018070 con 0x7f409c154670 2024-03-29T04:08:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a3b60640 1 -- 172.21.15.97:0/72359717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c13d800 con 0x7f409c154670 2024-03-29T04:08:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40927fc640 1 -- 172.21.15.97:0/72359717 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4098005490 con 0x7f409c154670 2024-03-29T04:08:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.949+0000 7f40a3b60640 1 -- 172.21.15.97:0/72359717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c13bd70 con 0x7f409c154670 2024-03-29T04:08:34.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.950+0000 7f40a3b60640 1 -- 172.21.15.97:0/72359717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409c09f6a0 con 0x7f409c154670 2024-03-29T04:08:34.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.951+0000 7f40927fc640 1 -- 172.21.15.97:0/72359717 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f409800d730 con 0x7f409c154670 2024-03-29T04:08:34.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.951+0000 7f40927fc640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f408006c560 0x7f408006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:34.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.951+0000 7f40a18fc640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f408006c560 0x7f408006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:34.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.951+0000 7f40927fc640 1 -- 172.21.15.97:0/72359717 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f409808b440 con 0x7f409c154670 2024-03-29T04:08:34.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.952+0000 7f40a18fc640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f408006c560 0x7f408006ea20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f408c034000 tx=0x7f408c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:34.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:34.956+0000 7f40927fc640 1 -- 172.21.15.97:0/72359717 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4098059af0 con 0x7f409c154670 2024-03-29T04:08:35.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.198+0000 7f40a3b60640 1 -- 172.21.15.97:0/72359717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f409c13eec0 con 0x7f409c154670 2024-03-29T04:08:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.201+0000 7f40927fc640 1 -- 172.21.15.97:0/72359717 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f40980593f0 con 0x7f409c154670 2024-03-29T04:08:35.180 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-03-29T04:08:35.180 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBTPgZm8VqYBhAAmwz9Zhd1EMjy52y5RAJy3w== 2024-03-29T04:08:35.181 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-03-29T04:08:35.181 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-03-29T04:08:35.181 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-29T04:08:35.181 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-03-29T04:08:35.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.203+0000 7f40a3b60640 1 -- 172.21.15.97:0/72359717 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f408006c560 msgr2=0x7f408006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:35.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.203+0000 7f40a3b60640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f408006c560 0x7f408006ea20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f408c034000 tx=0x7f408c009030 comp rx=0 tx=0).stop 2024-03-29T04:08:35.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.203+0000 7f40a3b60640 1 -- 172.21.15.97:0/72359717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f409c154670 msgr2=0x7f409c13e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:35.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.203+0000 7f40a3b60640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f409c154670 0x7f409c13e5a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f409800f820 tx=0x7f409800fce0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.203+0000 7f40a3b60640 1 -- 172.21.15.97:0/72359717 shutdown_connections 2024-03-29T04:08:35.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.203+0000 7f40a3b60640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f408006c560 0x7f408006ea20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.203+0000 7f40a3b60640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f409c13eae0 0x7f409c1f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.203+0000 7f40a3b60640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f409c154670 0x7f409c13e5a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.203+0000 7f40a3b60640 1 --2- 172.21.15.97:0/72359717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40880039b0 0x7f409c13b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.203+0000 7f40a3b60640 1 -- 172.21.15.97:0/72359717 >> 172.21.15.97:0/72359717 conn(0x7f409c072e40 msgr2=0x7f409c0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:35.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.203+0000 7f40a3b60640 1 -- 172.21.15.97:0/72359717 shutdown_connections 2024-03-29T04:08:35.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.204+0000 7f40a3b60640 1 -- 172.21.15.97:0/72359717 wait complete. 2024-03-29T04:08:35.193 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:08:35.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.314+0000 7f5babddb640 1 Processor -- start 2024-03-29T04:08:35.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.315+0000 7f5babddb640 1 -- start start 2024-03-29T04:08:35.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.315+0000 7f5babddb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ba415c790 0x7f5ba40d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.315+0000 7f5babddb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ba409b140 0x7f5ba40d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.315+0000 7f5babddb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ba415cb70 con 0x7f5ba40ce120 2024-03-29T04:08:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.315+0000 7f5babddb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f5ba409ba70 con 0x7f5ba415c790 2024-03-29T04:08:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.315+0000 7f5babddb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba40cdc90 con 0x7f5ba409b140 2024-03-29T04:08:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.315+0000 7f5ba9b77640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ba415c790 0x7f5ba40d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.316+0000 7f5ba9b77640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ba415c790 0x7f5ba40d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51300/0 (socket says 172.21.15.97:51300) 2024-03-29T04:08:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.316+0000 7f5ba9376640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5ba40ce120 0x7f5ba409aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55820/0 (socket says 172.21.15.97:55820) 2024-03-29T04:08:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.316+0000 7f5ba9b77640 1 -- 172.21.15.97:0/498839089 learned_addr learned my addr 172.21.15.97:0/498839089 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.316+0000 7f5baa378640 1 --2- 172.21.15.97:0/498839089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ba409b140 0x7f5ba40d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.316+0000 7f5ba9b77640 1 -- 172.21.15.97:0/498839089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ba409b140 msgr2=0x7f5ba40d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.316+0000 7f5ba9b77640 1 --2- 172.21.15.97:0/498839089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ba409b140 0x7f5ba40d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.316+0000 7f5ba9b77640 1 -- 172.21.15.97:0/498839089 >> v1:172.21.15.97:6789/0 conn(0x7f5ba40ce120 legacy=0x7f5ba409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.316+0000 7f5ba9b77640 1 -- 172.21.15.97:0/498839089 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba40cd080 con 0x7f5ba415c790 2024-03-29T04:08:35.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.316+0000 7f5ba8b75640 1 -- 172.21.15.97:0/498839089 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1815111344 0 0) 0x7f5ba415cb70 con 0x7f5ba40ce120 2024-03-29T04:08:35.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.317+0000 7f5ba9b77640 1 --2- 172.21.15.97:0/498839089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ba415c790 0x7f5ba40d09f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009870 tx=0x7f5ba002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c7c2815c47d86b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:35.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.317+0000 7f5ba8b75640 1 -- 172.21.15.97:0/498839089 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0034070 con 0x7f5ba415c790 2024-03-29T04:08:35.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.317+0000 7f5ba8b75640 1 -- 172.21.15.97:0/498839089 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba0030840 con 0x7f5ba415c790 2024-03-29T04:08:35.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.317+0000 7f5ba8b75640 1 -- 172.21.15.97:0/498839089 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0030c10 con 0x7f5ba415c790 2024-03-29T04:08:35.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.317+0000 7f5babddb640 1 -- 172.21.15.97:0/498839089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ba415c790 msgr2=0x7f5ba40d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:35.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.317+0000 7f5babddb640 1 --2- 172.21.15.97:0/498839089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ba415c790 0x7f5ba40d09f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009870 tx=0x7f5ba002b980 comp rx=0 tx=0).stop 2024-03-29T04:08:35.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.317+0000 7f5babddb640 1 -- 172.21.15.97:0/498839089 shutdown_connections 2024-03-29T04:08:35.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.317+0000 7f5babddb640 1 --2- 172.21.15.97:0/498839089 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ba409b140 0x7f5ba40d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.317+0000 7f5babddb640 1 --2- 172.21.15.97:0/498839089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ba415c790 0x7f5ba40d09f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.317+0000 7f5babddb640 1 -- 172.21.15.97:0/498839089 >> 172.21.15.97:0/498839089 conn(0x7f5ba40b3f00 msgr2=0x7f5ba40b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:35.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.317+0000 7f5babddb640 1 -- 172.21.15.97:0/498839089 shutdown_connections 2024-03-29T04:08:35.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.317+0000 7f5babddb640 1 -- 172.21.15.97:0/498839089 wait complete. 2024-03-29T04:08:35.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.318+0000 7f5babddb640 1 Processor -- start 2024-03-29T04:08:35.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.318+0000 7f5babddb640 1 -- start start 2024-03-29T04:08:35.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.319+0000 7f5babddb640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ba409b140 0x7f5ba41f1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:35.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.319+0000 7f5babddb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ba40ce120 0x7f5ba41f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:35.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.319+0000 7f5babddb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba415c790 0x7f5ba406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:35.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.319+0000 7f5babddb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba40cd920 con 0x7f5ba40ce120 2024-03-29T04:08:35.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.319+0000 7f5babddb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba40cd640 con 0x7f5ba415c790 2024-03-29T04:08:35.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.319+0000 7f5babddb640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba40b1d00 con 0x7f5ba409b140 2024-03-29T04:08:35.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.319+0000 7f5ba9376640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ba40ce120 0x7f5ba41f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.319+0000 7f5ba9376640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ba40ce120 0x7f5ba41f4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51310/0 (socket says 172.21.15.97:51310) 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.319+0000 7f5ba9376640 1 -- 172.21.15.97:0/24199219 learned_addr learned my addr 172.21.15.97:0/24199219 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.319+0000 7f5ba9b77640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ba409b140 0x7f5ba41f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.319+0000 7f5baa378640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba415c790 0x7f5ba406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.320+0000 7f5ba9376640 1 -- 172.21.15.97:0/24199219 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ba409b140 msgr2=0x7f5ba41f1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.320+0000 7f5ba9376640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ba409b140 0x7f5ba41f1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.320+0000 7f5ba9376640 1 -- 172.21.15.97:0/24199219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba415c790 msgr2=0x7f5ba406cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.320+0000 7f5ba9376640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba415c790 0x7f5ba406cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.320+0000 7f5ba9376640 1 -- 172.21.15.97:0/24199219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba406c620 con 0x7f5ba40ce120 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.320+0000 7f5ba9b77640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ba409b140 0x7f5ba41f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.320+0000 7f5baa378640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba415c790 0x7f5ba406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.320+0000 7f5ba9376640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ba40ce120 0x7f5ba41f4040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400c4d0 tx=0x7f5b9400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.320+0000 7f5b9a7fc640 1 -- 172.21.15.97:0/24199219 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b94018070 con 0x7f5ba40ce120 2024-03-29T04:08:35.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.320+0000 7f5b9a7fc640 1 -- 172.21.15.97:0/24199219 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b94007500 con 0x7f5ba40ce120 2024-03-29T04:08:35.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.320+0000 7f5b9a7fc640 1 -- 172.21.15.97:0/24199219 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b94004a20 con 0x7f5ba40ce120 2024-03-29T04:08:35.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.320+0000 7f5babddb640 1 -- 172.21.15.97:0/24199219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba41f4900 con 0x7f5ba40ce120 2024-03-29T04:08:35.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.320+0000 7f5babddb640 1 -- 172.21.15.97:0/24199219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba41f4ec0 con 0x7f5ba40ce120 2024-03-29T04:08:35.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.321+0000 7f5babddb640 1 -- 172.21.15.97:0/24199219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba4068d50 con 0x7f5ba40ce120 2024-03-29T04:08:35.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.321+0000 7f5b9a7fc640 1 -- 172.21.15.97:0/24199219 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5b9401b070 con 0x7f5ba40ce120 2024-03-29T04:08:35.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.322+0000 7f5b9a7fc640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5b8806c530 0x7f5b8806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:35.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.322+0000 7f5b9a7fc640 1 -- 172.21.15.97:0/24199219 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5b9408aa00 con 0x7f5ba40ce120 2024-03-29T04:08:35.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.322+0000 7f5ba9b77640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5b8806c530 0x7f5b8806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:35.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.322+0000 7f5ba9b77640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5b8806c530 0x7f5b8806e9f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0036000 tx=0x7f5ba0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:35.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.326+0000 7f5b9a7fc640 1 -- 172.21.15.97:0/24199219 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5b94059020 con 0x7f5ba40ce120 2024-03-29T04:08:35.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.547+0000 7f5babddb640 1 -- 172.21.15.97:0/24199219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ba406ea30 con 0x7f5ba40ce120 2024-03-29T04:08:35.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.547+0000 7f5b9a7fc640 1 -- 172.21.15.97:0/24199219 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5184 (secure 0 0 0) 0x7f5b94058920 con 0x7f5ba40ce120 2024-03-29T04:08:35.527 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:08:35.527 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2199898540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2199898540},{"type":"v1","addr":"172.21.15.97:6837","nonce":2199898540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1223352413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1223352413},{"type":"v1","addr":"172.21.15.154:6834","nonce":1223352413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/830866132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":830866132},{"type":"v1","addr":"172.21.15.154:6835","nonce":830866132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:08:29.955858+0000","modified":"2024-03-29T04:08:32.584967+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5067":{"gid":5067,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/3142898446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3142898446},{"type":"v1","addr":"172.21.15.97:6835","nonce":3142898446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5067,"qdb_cluster":[5067]},"id":2}]} 2024-03-29T04:08:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 14 2024-03-29T04:08:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.549+0000 7f5babddb640 1 -- 172.21.15.97:0/24199219 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5b8806c530 msgr2=0x7f5b8806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.549+0000 7f5babddb640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5b8806c530 0x7f5b8806e9f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0036000 tx=0x7f5ba0008030 comp rx=0 tx=0).stop 2024-03-29T04:08:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.549+0000 7f5babddb640 1 -- 172.21.15.97:0/24199219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ba40ce120 msgr2=0x7f5ba41f4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.549+0000 7f5babddb640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ba40ce120 0x7f5ba41f4040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400c4d0 tx=0x7f5b9400c990 comp rx=0 tx=0).stop 2024-03-29T04:08:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.549+0000 7f5babddb640 1 -- 172.21.15.97:0/24199219 shutdown_connections 2024-03-29T04:08:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.549+0000 7f5babddb640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5b8806c530 0x7f5b8806e9f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.549+0000 7f5babddb640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba415c790 0x7f5ba406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.549+0000 7f5babddb640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ba40ce120 0x7f5ba41f4040 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.549+0000 7f5babddb640 1 --2- 172.21.15.97:0/24199219 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ba409b140 0x7f5ba41f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:35.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.549+0000 7f5babddb640 1 -- 172.21.15.97:0/24199219 >> 172.21.15.97:0/24199219 conn(0x7f5ba40b3f00 msgr2=0x7f5ba40b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:35.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.549+0000 7f5babddb640 1 -- 172.21.15.97:0/24199219 shutdown_connections 2024-03-29T04:08:35.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:35.549+0000 7f5babddb640 1 -- 172.21.15.97:0/24199219 wait complete. 2024-03-29T04:08:35.943 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T04:08:29.955858+0000', 'modified': '2024-03-29T04:08:32.584967+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5067}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5067': {'gid': 5067, 'name': 'c', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6835/3142898446', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 3142898446}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 3142898446}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5067, 'qdb_cluster': [5067]} 2024-03-29T04:08:35.943 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T04:08:35.943 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-29T04:08:36.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.065+0000 7f4ea6e7b640 1 Processor -- start 2024-03-29T04:08:36.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.065+0000 7f4ea6e7b640 1 -- start start 2024-03-29T04:08:36.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.066+0000 7f4ea6e7b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ea015c740 0x7f4ea00cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.066+0000 7f4ea6e7b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea00cceb0 0x7f4ea00d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.066+0000 7f4ea6e7b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ea00d1310 con 0x7f4ea00d0960 2024-03-29T04:08:36.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.066+0000 7f4ea6e7b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea00cd310 con 0x7f4ea015c740 2024-03-29T04:08:36.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.066+0000 7f4ea6e7b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea009b850 con 0x7f4ea00cceb0 2024-03-29T04:08:36.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.066+0000 7f4ea5418640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4ea00d0960 0x7f4ea0160180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55830/0 (socket says 172.21.15.97:55830) 2024-03-29T04:08:36.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.066+0000 7f4ea5418640 1 -- 172.21.15.97:0/3450679073 learned_addr learned my addr 172.21.15.97:0/3450679073 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.066+0000 7f4e9ffff640 1 --2- 172.21.15.97:0/3450679073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea00cceb0 0x7f4ea00d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.066+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/3450679073 >> v2:172.21.15.154:3300/0 conn(0x7f4ea015c740 0x7f4ea00cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.066+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 368265373 0 0) 0x7f4ea00d1310 con 0x7f4ea00d0960 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.066+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e90003520 con 0x7f4ea00d0960 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 868338245 0 0) 0x7f4e90003520 con 0x7f4ea00d0960 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea00cceb0 msgr2=0x7f4ea00d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 --2- 172.21.15.97:0/3450679073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea00cceb0 0x7f4ea00d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 >> v2:172.21.15.154:3300/0 conn(0x7f4ea015c740 msgr2=0x7f4ea00cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 --2- 172.21.15.97:0/3450679073 >> v2:172.21.15.154:3300/0 conn(0x7f4ea015c740 0x7f4ea00cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea0070a20 con 0x7f4ea00d0960 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f4e980019f0 con 0x7f4ea00d0960 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 >> v1:172.21.15.97:6789/0 conn(0x7f4ea00d0960 legacy=0x7f4ea0160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 --2- 172.21.15.97:0/3450679073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea00cceb0 0x7f4e900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 --2- 172.21.15.97:0/3450679073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e90003f30 0x7f4e90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 --2- 172.21.15.97:0/3450679073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e90004d40 0x7f4e900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea00cd310 con 0x7f4e90004d40 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea009b850 con 0x7f4e90003f30 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4e9000b9c0 con 0x7f4ea00cceb0 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4ea5418640 1 --2- 172.21.15.97:0/3450679073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e90004d40 0x7f4e900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4e9ffff640 1 --2- 172.21.15.97:0/3450679073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e90003f30 0x7f4e90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/3450679073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea00cceb0 0x7f4e900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4ea5418640 1 -- 172.21.15.97:0/3450679073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea00cceb0 msgr2=0x7f4e900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4ea5418640 1 --2- 172.21.15.97:0/3450679073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea00cceb0 0x7f4e900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4ea5418640 1 -- 172.21.15.97:0/3450679073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e90003f30 msgr2=0x7f4e90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4ea5418640 1 --2- 172.21.15.97:0/3450679073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e90003f30 0x7f4e90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.067+0000 7f4ea5418640 1 -- 172.21.15.97:0/3450679073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e90003520 con 0x7f4e90004d40 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.068+0000 7f4ea5418640 1 --2- 172.21.15.97:0/3450679073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e90004d40 0x7f4e900051c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4e980013d0 tx=0x7f4e90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad863ec5db2486c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.069+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800a4e0 con 0x7f4e90004d40 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.069+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e98034650 con 0x7f4e90004d40 2024-03-29T04:08:36.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.069+0000 7f4e9f7fe640 1 -- 172.21.15.97:0/3450679073 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98034a40 con 0x7f4e90004d40 2024-03-29T04:08:36.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.069+0000 7f4ea6e7b640 1 -- 172.21.15.97:0/3450679073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e90004d40 msgr2=0x7f4e900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.069+0000 7f4ea6e7b640 1 --2- 172.21.15.97:0/3450679073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e90004d40 0x7f4e900051c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4e980013d0 tx=0x7f4e90005830 comp rx=0 tx=0).stop 2024-03-29T04:08:36.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.069+0000 7f4ea5418640 1 -- 172.21.15.97:0/3450679073 reap_dead start 2024-03-29T04:08:36.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.069+0000 7f4ea6e7b640 1 -- 172.21.15.97:0/3450679073 shutdown_connections 2024-03-29T04:08:36.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.069+0000 7f4ea6e7b640 1 -- 172.21.15.97:0/3450679073 >> 172.21.15.97:0/3450679073 conn(0x7f4ea00b3e90 msgr2=0x7f4ea00b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:36.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.069+0000 7f4ea6e7b640 1 -- 172.21.15.97:0/3450679073 shutdown_connections 2024-03-29T04:08:36.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.069+0000 7f4ea6e7b640 1 -- 172.21.15.97:0/3450679073 wait complete. 2024-03-29T04:08:36.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.070+0000 7f4ea6e7b640 1 Processor -- start 2024-03-29T04:08:36.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.070+0000 7f4ea6e7b640 1 -- start start 2024-03-29T04:08:36.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.071+0000 7f4ea6e7b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ea00b55b0 0x7f4ea00b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.071+0000 7f4ea6e7b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea00b5990 0x7f4ea01f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.071+0000 7f4ea6e7b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea00b78b0 0x7f4ea01f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.071+0000 7f4ea6e7b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea009b200 con 0x7f4ea00b5990 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.071+0000 7f4ea6e7b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea009af20 con 0x7f4ea00b55b0 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.071+0000 7f4ea6e7b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea009b4e0 con 0x7f4ea00b78b0 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.071+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea00b5990 0x7f4ea01f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.071+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea00b5990 0x7f4ea01f21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51324/0 (socket says 172.21.15.97:51324) 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.071+0000 7f4e9ffff640 1 -- 172.21.15.97:0/2603161756 learned_addr learned my addr 172.21.15.97:0/2603161756 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.072+0000 7f4e9ffff640 1 -- 172.21.15.97:0/2603161756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea00b78b0 msgr2=0x7f4ea01f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.072+0000 7f4e9ffff640 1 --2- 172.21.15.97:0/2603161756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea00b78b0 0x7f4ea01f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.072+0000 7f4e9ffff640 1 -- 172.21.15.97:0/2603161756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ea00b55b0 msgr2=0x7f4ea00b7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.072+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/2603161756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ea00b55b0 0x7f4ea00b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.072+0000 7f4e9ffff640 1 --2- 172.21.15.97:0/2603161756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ea00b55b0 0x7f4ea00b7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.072+0000 7f4e9ffff640 1 -- 172.21.15.97:0/2603161756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea00b7e50 con 0x7f4ea00b5990 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.072+0000 7f4e9ffff640 1 --2- 172.21.15.97:0/2603161756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea00b5990 0x7f4ea01f21b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0118e0 tx=0x7f4e8c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:36.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.072+0000 7f4e9d7fa640 1 -- 172.21.15.97:0/2603161756 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00fa30 con 0x7f4ea00b5990 2024-03-29T04:08:36.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.072+0000 7f4e9d7fa640 1 -- 172.21.15.97:0/2603161756 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e8c00fbf0 con 0x7f4ea00b5990 2024-03-29T04:08:36.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.072+0000 7f4ea6e7b640 1 -- 172.21.15.97:0/2603161756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea00b8130 con 0x7f4ea00b5990 2024-03-29T04:08:36.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.072+0000 7f4e9d7fa640 1 -- 172.21.15.97:0/2603161756 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c014440 con 0x7f4ea00b5990 2024-03-29T04:08:36.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.072+0000 7f4ea6e7b640 1 -- 172.21.15.97:0/2603161756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea014a580 con 0x7f4ea00b5990 2024-03-29T04:08:36.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.073+0000 7f4e9d7fa640 1 -- 172.21.15.97:0/2603161756 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4e8c012040 con 0x7f4ea00b5990 2024-03-29T04:08:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.074+0000 7f4e9d7fa640 1 --2- 172.21.15.97:0/2603161756 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4e8406c530 0x7f4e8406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.074+0000 7f4e9d7fa640 1 -- 172.21.15.97:0/2603161756 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4e8c08be00 con 0x7f4ea00b5990 2024-03-29T04:08:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.074+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/2603161756 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4e8406c530 0x7f4e8406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.074+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/2603161756 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4e8406c530 0x7f4e8406e9f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009a60 tx=0x7f4e94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:36.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.075+0000 7f4ea9a6d740 1 Processor -- start 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.075+0000 7f4ea9a6d740 1 -- start start 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea9a6d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf1b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea9a6d740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 0x55ffcdf4b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea9a6d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf1bd40 0x55ffcdf4bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea9a6d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ffcde85020 con 0x55ffcdf1bd40 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea9a6d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ffcde2d5f0 con 0x55ffcdf48860 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea9a6d740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55ffcde80fd0 con 0x55ffcdf48c40 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea5418640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf1bd40 0x55ffcdf4bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea5418640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf1bd40 0x55ffcdf4bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51334/0 (socket says 172.21.15.97:51334) 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea5418640 1 -- 172.21.15.97:0/2126328578 learned_addr learned my addr 172.21.15.97:0/2126328578 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea5418640 1 -- 172.21.15.97:0/2126328578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 msgr2=0x55ffcdf4b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4e9ffff640 1 --2- 172.21.15.97:0/2126328578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 0x55ffcdf4b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/2126328578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf1b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea5418640 1 --2- 172.21.15.97:0/2126328578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 0x55ffcdf4b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea5418640 1 -- 172.21.15.97:0/2126328578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 msgr2=0x55ffcdf1b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea5418640 1 --2- 172.21.15.97:0/2126328578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf1b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea5418640 1 -- 172.21.15.97:0/2126328578 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffcdb992d0 con 0x55ffcdf1bd40 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/2126328578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf1b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4ea5418640 1 --2- 172.21.15.97:0/2126328578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf1bd40 0x55ffcdf4bb30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800a0b0 tx=0x7f4e9802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff28139743b8232 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4e8a7fc640 1 -- 172.21.15.97:0/2126328578 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98002040 con 0x55ffcdf1bd40 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4e8a7fc640 1 -- 172.21.15.97:0/2126328578 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e98042070 con 0x55ffcdf1bd40 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.076+0000 7f4e8a7fc640 1 -- 172.21.15.97:0/2126328578 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e980017b0 con 0x55ffcdf1bd40 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.077+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/2126328578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf1bd40 msgr2=0x55ffcdf4bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.077+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/2126328578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf1bd40 0x55ffcdf4bb30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800a0b0 tx=0x7f4e9802e000 comp rx=0 tx=0).stop 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.077+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/2126328578 shutdown_connections 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.077+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/2126328578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf1bd40 0x55ffcdf4bb30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.077+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/2126328578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 0x55ffcdf4b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.077+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/2126328578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf1b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.077+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/2126328578 >> 172.21.15.97:0/2126328578 conn(0x55ffcdf1da70 msgr2=0x55ffcdf665f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:36.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.077+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/2126328578 shutdown_connections 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.077+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/2126328578 wait complete. 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.078+0000 7f4ea9a6d740 1 -- start start 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.078+0000 7f4ea9a6d740 1 Processor -- start 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4ea9a6d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf66c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4ea9a6d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48c40 0x55ffcdf40f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4ea9a6d740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf4f030 0x55ffcdf4d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4ea9a6d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ffcdf20110 con 0x55ffcdf48c40 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4ea9a6d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ffcde409c0 con 0x55ffcdf48860 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4ea9a6d740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55ffcdf22950 con 0x55ffcdf4f030 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48c40 0x55ffcdf40f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48c40 0x55ffcdf40f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51348/0 (socket says 172.21.15.97:51348) 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4e9ffff640 1 -- 172.21.15.97:0/1537906721 learned_addr learned my addr 172.21.15.97:0/1537906721 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4ea9a6d740 10 client.0 _ll_register_callbacks cb 0x55ffcde24fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf66c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4ea5418640 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf4f030 0x55ffcdf4d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4e9ffff640 1 -- 172.21.15.97:0/1537906721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf4f030 msgr2=0x55ffcdf4d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4e9ffff640 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf4f030 0x55ffcdf4d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4e9ffff640 1 -- 172.21.15.97:0/1537906721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 msgr2=0x55ffcdf66c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4e9ffff640 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf66c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4e9ffff640 1 -- 172.21.15.97:0/1537906721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffcdc636b0 con 0x55ffcdf48c40 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf66c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4e9ffff640 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48c40 0x55ffcdf40f90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00f610 tx=0x7f4e8c05e210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05b41f7ad2680b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4e7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4e7f7fe640 1 -- 172.21.15.97:0/1537906721 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c072020 con 0x55ffcdf48c40 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4e7f7fe640 1 -- 172.21.15.97:0/1537906721 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4e8c062470 con 0x55ffcdf48c40 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/1537906721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e98042070 con 0x55ffcdf48c40 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4e7f7fe640 1 -- 172.21.15.97:0/1537906721 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f4e8c062670 con 0x55ffcdf48c40 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4ea9a6d740 10 client.5259 fetch_fsmap learned FSMap version 14 2024-03-29T04:08:36.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.079+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/1537906721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55ffcdf66660 con 0x55ffcdf48c40 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.080+0000 7f4e7f7fe640 1 -- 172.21.15.97:0/1537906721 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e98042070 con 0x55ffcdf48c40 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.080+0000 7f4e7f7fe640 1 -- 172.21.15.97:0/1537906721 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7f4e8c059220 con 0x55ffcdf48c40 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.080+0000 7f4ea9a6d740 10 client.5259 fetch_fsmap finished waiting for FSMap version 14 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.080+0000 7f4ea9a6d740 10 client.5259 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.080+0000 7f4ea9a6d740 20 client.5259 populate_metadata read hostname 'smithi097' 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.080+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ffcdf7a400 0x55ffcdf4a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.080+0000 7f4ea9a6d740 4 client.5259 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.080+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/1537906721 --> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffcde594e0 con 0x55ffcdf7a400 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.080+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ffcdf7a400 0x55ffcdf4a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.080+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ffcdf7a400 0x55ffcdf4a610 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9367543afbfaeccc server_cookie=bed1dd198cd10871 in_seq=0 out_seq=0 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.080+0000 7f4e7f7fe640 10 client.5259 ms_handle_connect on v2:172.21.15.97:6834/3142898446 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.081+0000 7f4e7f7fe640 1 -- 172.21.15.97:0/1537906721 <== mds.0 v2:172.21.15.97:6834/3142898446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55ffcde594e0 con 0x55ffcdf7a400 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.081+0000 7f4e7f7fe640 10 client.5259 handle_command_reply: tid=0 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.081+0000 7f4ea9a6d740 1 client.5259 shutdown 2024-03-29T04:08:36.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.081+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/1537906721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48c40 msgr2=0x55ffcdf40f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.081+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48c40 0x55ffcdf40f90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00f610 tx=0x7f4e8c05e210 comp rx=0 tx=0).stop 2024-03-29T04:08:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.082+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/1537906721 shutdown_connections 2024-03-29T04:08:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.082+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ffcdf7a400 0x55ffcdf4a610 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.082+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf4f030 0x55ffcdf4d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.082+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48c40 0x55ffcdf40f90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.082+0000 7f4e7f7fe640 0 client.5259 ms_handle_reset on v2:172.21.15.97:6834/3142898446 2024-03-29T04:08:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.082+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/1537906721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf66c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.082+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/1537906721 >> 172.21.15.97:0/1537906721 conn(0x55ffcdf1bd40 msgr2=0x55ffcdf1f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.082+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/1537906721 shutdown_connections 2024-03-29T04:08:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.082+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/1537906721 wait complete. 2024-03-29T04:08:36.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.082+0000 7f4ea9a6d740 20 client.5259 trim_cache size 0 max 16384 2024-03-29T04:08:36.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.100+0000 7f4ea9a6d740 1 Processor -- start 2024-03-29T04:08:36.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.100+0000 7f4ea9a6d740 1 -- start start 2024-03-29T04:08:36.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea9a6d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48860 0x55ffcdf492d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea9a6d740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 0x55ffcdf64a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea9a6d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf4f030 0x55ffcdf64f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea9a6d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea01fe6c0 con 0x55ffcdf48860 2024-03-29T04:08:36.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea9a6d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea00cadd0 con 0x55ffcdf4f030 2024-03-29T04:08:36.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea9a6d740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea01fe2b0 con 0x55ffcdf48c40 2024-03-29T04:08:36.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea4c17640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48860 0x55ffcdf492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea4c17640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48860 0x55ffcdf492d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51350/0 (socket says 172.21.15.97:51350) 2024-03-29T04:08:36.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea4c17640 1 -- 172.21.15.97:0/247934566 learned_addr learned my addr 172.21.15.97:0/247934566 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:36.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4e9ffff640 1 --2- 172.21.15.97:0/247934566 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 0x55ffcdf64a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea4c17640 1 -- 172.21.15.97:0/247934566 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 msgr2=0x55ffcdf64a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/247934566 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 0x55ffcdf64a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea4c17640 1 -- 172.21.15.97:0/247934566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf4f030 msgr2=0x55ffcdf64f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/247934566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf4f030 0x55ffcdf64f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea4c17640 1 -- 172.21.15.97:0/247934566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffcdf40770 con 0x55ffcdf48860 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/247934566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48860 0x55ffcdf492d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4e9402bc80 tx=0x7f4e94016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1634890b4d50d841 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4e89ffb640 1 -- 172.21.15.97:0/247934566 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94031070 con 0x55ffcdf48860 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4e89ffb640 1 -- 172.21.15.97:0/247934566 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e9401c040 con 0x55ffcdf48860 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.101+0000 7f4e89ffb640 1 -- 172.21.15.97:0/247934566 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9402bcb0 con 0x55ffcdf48860 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.102+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/247934566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48860 msgr2=0x55ffcdf492d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.102+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/247934566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48860 0x55ffcdf492d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4e9402bc80 tx=0x7f4e94016000 comp rx=0 tx=0).stop 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.102+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/247934566 shutdown_connections 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.102+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/247934566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf4f030 0x55ffcdf64f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.102+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/247934566 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 0x55ffcdf64a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.102+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/247934566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf48860 0x55ffcdf492d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.102+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/247934566 >> 172.21.15.97:0/247934566 conn(0x55ffcdf1bd40 msgr2=0x55ffcdf22340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:36.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.102+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/247934566 shutdown_connections 2024-03-29T04:08:36.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.102+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/247934566 wait complete. 2024-03-29T04:08:36.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.102+0000 7f4ea9a6d740 1 -- start start 2024-03-29T04:08:36.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.103+0000 7f4ea9a6d740 1 Processor -- start 2024-03-29T04:08:36.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.103+0000 7f4ea9a6d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.103+0000 7f4ea9a6d740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 0x55ffcdf67cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.103+0000 7f4ea9a6d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf4f030 0x55ffcdf89830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.103+0000 7f4ea9a6d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ffcdf7a280 con 0x55ffcdf4f030 2024-03-29T04:08:36.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.103+0000 7f4ea9a6d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ffcdf4b2d0 con 0x55ffcdf48860 2024-03-29T04:08:36.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.103+0000 7f4ea9a6d740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55ffcdf7a870 con 0x55ffcdf48c40 2024-03-29T04:08:36.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.103+0000 7f4ea9a6d740 10 client.0 _ll_register_callbacks cb 0x7f4e8c012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:08:36.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.103+0000 7f4ea5418640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf4f030 0x55ffcdf89830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.103+0000 7f4ea5418640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf4f030 0x55ffcdf89830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51360/0 (socket says 172.21.15.97:51360) 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4ea5418640 1 -- 172.21.15.97:0/4016031536 learned_addr learned my addr 172.21.15.97:0/4016031536 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4e9ffff640 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 0x55ffcdf67cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4ea5418640 1 -- 172.21.15.97:0/4016031536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 msgr2=0x55ffcdf67cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4ea5418640 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 0x55ffcdf67cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4ea5418640 1 -- 172.21.15.97:0/4016031536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 msgr2=0x55ffcdf677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4ea5418640 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf677b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4ea5418640 1 -- 172.21.15.97:0/4016031536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffcdf68e40 con 0x55ffcdf4f030 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4e9ffff640 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 0x55ffcdf67cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf677b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4ea5418640 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf4f030 0x55ffcdf89830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4e9803f040 tx=0x7f4e98001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76d19d639c4b2c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4e7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4e7f7fe640 1 -- 172.21.15.97:0/4016031536 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800a4c0 con 0x55ffcdf4f030 2024-03-29T04:08:36.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4e7f7fe640 1 -- 172.21.15.97:0/4016031536 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4e980017b0 con 0x55ffcdf4f030 2024-03-29T04:08:36.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/4016031536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9401c040 con 0x55ffcdf4f030 2024-03-29T04:08:36.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4e7f7fe640 1 -- 172.21.15.97:0/4016031536 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f4e98057050 con 0x55ffcdf4f030 2024-03-29T04:08:36.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4ea9a6d740 10 client.5265 fetch_fsmap learned FSMap version 14 2024-03-29T04:08:36.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/4016031536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55ffcdf8a450 con 0x55ffcdf4f030 2024-03-29T04:08:36.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4e7f7fe640 1 -- 172.21.15.97:0/4016031536 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e9401c040 con 0x55ffcdf4f030 2024-03-29T04:08:36.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.104+0000 7f4e7f7fe640 1 -- 172.21.15.97:0/4016031536 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 14) ==== 3494+0+0 (secure 0 0 0) 0x7f4e9805a020 con 0x55ffcdf4f030 2024-03-29T04:08:36.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 10 client.5265 fetch_fsmap finished waiting for FSMap version 14 2024-03-29T04:08:36.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 10 client.5265 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T04:08:36.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 20 client.5265 populate_metadata read hostname 'smithi097' 2024-03-29T04:08:36.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ffcdf78c60 0x55ffcdf79040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 4 client.5265 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T04:08:36.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/4016031536 --> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] -- command(tid 0: {"prefix": "status"}) -- 0x55ffcdf4eba0 con 0x55ffcdf78c60 2024-03-29T04:08:36.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ffcdf78c60 0x55ffcdf79040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4e7f7fe640 10 client.5265 ms_handle_connect on v2:172.21.15.97:6834/3142898446 2024-03-29T04:08:36.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea4c17640 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ffcdf78c60 0x55ffcdf79040 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6f77f07e98e88c5 server_cookie=9c0a5698654f7d2f in_seq=0 out_seq=0 2024-03-29T04:08:36.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4e7f7fe640 1 -- 172.21.15.97:0/4016031536 <== mds.0 v2:172.21.15.97:6834/3142898446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ffcdf4eba0 con 0x55ffcdf78c60 2024-03-29T04:08:36.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4e7f7fe640 10 client.5265 handle_command_reply: tid=0 2024-03-29T04:08:36.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 1 client.5265 shutdown 2024-03-29T04:08:36.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/4016031536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf4f030 msgr2=0x55ffcdf89830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf4f030 0x55ffcdf89830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4e9803f040 tx=0x7f4e98001360 comp rx=0 tx=0).stop 2024-03-29T04:08:36.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/4016031536 shutdown_connections 2024-03-29T04:08:36.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ffcdf78c60 0x55ffcdf79040 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffcdf4f030 0x55ffcdf89830 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ffcdf48c40 0x55ffcdf67cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4e7f7fe640 0 client.5265 ms_handle_reset on v2:172.21.15.97:6834/3142898446 2024-03-29T04:08:36.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 1 --2- 172.21.15.97:0/4016031536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ffcdf48860 0x55ffcdf677b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/4016031536 >> 172.21.15.97:0/4016031536 conn(0x55ffcdf1bd40 msgr2=0x55ffcdf76330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:36.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.105+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/4016031536 shutdown_connections 2024-03-29T04:08:36.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea9a6d740 1 -- 172.21.15.97:0/4016031536 wait complete. 2024-03-29T04:08:36.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea9a6d740 20 client.5265 trim_cache size 0 max 16384 2024-03-29T04:08:36.092 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:08:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f", 2024-03-29T04:08:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-03-29T04:08:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5067, 2024-03-29T04:08:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-03-29T04:08:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-03-29T04:08:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-03-29T04:08:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 5.5032010939999996, 2024-03-29T04:08:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 14, 2024-03-29T04:08:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 32, 2024-03-29T04:08:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 32, 2024-03-29T04:08:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 12.388495110999999 2024-03-29T04:08:36.093 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:08:36.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea6e7b640 1 -- 172.21.15.97:0/2603161756 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4e8406c530 msgr2=0x7f4e8406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea6e7b640 1 --2- 172.21.15.97:0/2603161756 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4e8406c530 0x7f4e8406e9f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009a60 tx=0x7f4e94008040 comp rx=0 tx=0).stop 2024-03-29T04:08:36.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea6e7b640 1 -- 172.21.15.97:0/2603161756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea00b5990 msgr2=0x7f4ea01f21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea6e7b640 1 --2- 172.21.15.97:0/2603161756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea00b5990 0x7f4ea01f21b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0118e0 tx=0x7f4e8c011da0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea6e7b640 1 -- 172.21.15.97:0/2603161756 shutdown_connections 2024-03-29T04:08:36.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea6e7b640 1 --2- 172.21.15.97:0/2603161756 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4e8406c530 0x7f4e8406e9f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea6e7b640 1 --2- 172.21.15.97:0/2603161756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ea00b78b0 0x7f4ea01f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea6e7b640 1 --2- 172.21.15.97:0/2603161756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea00b5990 0x7f4ea01f21b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea6e7b640 1 --2- 172.21.15.97:0/2603161756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ea00b55b0 0x7f4ea00b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea6e7b640 1 -- 172.21.15.97:0/2603161756 >> 172.21.15.97:0/2603161756 conn(0x7f4ea00b3e90 msgr2=0x7f4ea00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:36.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea6e7b640 1 -- 172.21.15.97:0/2603161756 shutdown_connections 2024-03-29T04:08:36.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.106+0000 7f4ea6e7b640 1 -- 172.21.15.97:0/2603161756 wait complete. 2024-03-29T04:08:36.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:08:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.217+0000 7f647930e640 1 Processor -- start 2024-03-29T04:08:36.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.217+0000 7f647930e640 1 -- start start 2024-03-29T04:08:36.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.218+0000 7f647930e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64740d1100 0x7f647409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.218+0000 7f647930e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f647415cb70 con 0x7f64740ce120 2024-03-29T04:08:36.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.218+0000 7f647930e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64740cd080 con 0x7f647415c790 2024-03-29T04:08:36.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.218+0000 7f647930e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f647409ba70 con 0x7f64740d1100 2024-03-29T04:08:36.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.218+0000 7f64737fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f64740ce120 0x7f64740b1d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55838/0 (socket says 172.21.15.97:55838) 2024-03-29T04:08:36.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.218+0000 7f64737fe640 1 -- 172.21.15.97:0/3586602880 learned_addr learned my addr 172.21.15.97:0/3586602880 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:36.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.218+0000 7f64727fc640 1 --2- 172.21.15.97:0/3586602880 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64740d1100 0x7f647409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.218+0000 7f6471ffb640 1 -- 172.21.15.97:0/3586602880 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 994908948 0 0) 0x7f647415cb70 con 0x7f64740ce120 2024-03-29T04:08:36.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.218+0000 7f6471ffb640 1 -- 172.21.15.97:0/3586602880 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f645c003520 con 0x7f64740ce120 2024-03-29T04:08:36.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.219+0000 7f6471ffb640 1 -- 172.21.15.97:0/3586602880 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 52284042 0 0) 0x7f64740cd080 con 0x7f647415c790 2024-03-29T04:08:36.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.219+0000 7f6471ffb640 1 -- 172.21.15.97:0/3586602880 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f647415cb70 con 0x7f647415c790 2024-03-29T04:08:36.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.219+0000 7f64727fc640 1 -- 172.21.15.97:0/3586602880 >> v1:172.21.15.154:6789/0 conn(0x7f647415c790 legacy=0x7f64740d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.219+0000 7f64727fc640 1 -- 172.21.15.97:0/3586602880 >> v1:172.21.15.97:6789/0 conn(0x7f64740ce120 legacy=0x7f64740b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.219+0000 7f6471ffb640 1 -- 172.21.15.97:0/3586602880 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3283175033 0 0) 0x7f645c003520 con 0x7f64740ce120 2024-03-29T04:08:36.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.219+0000 7f64727fc640 1 -- 172.21.15.97:0/3586602880 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64740cd360 con 0x7f64740d1100 2024-03-29T04:08:36.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.219+0000 7f64727fc640 1 --2- 172.21.15.97:0/3586602880 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64740d1100 0x7f647409aa30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f64680097d0 tx=0x7f646802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e68c398e6624342 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:36.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.220+0000 7f6471ffb640 1 -- 172.21.15.97:0/3586602880 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468037070 con 0x7f64740d1100 2024-03-29T04:08:36.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.220+0000 7f6471ffb640 1 -- 172.21.15.97:0/3586602880 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6468030510 con 0x7f64740d1100 2024-03-29T04:08:36.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.220+0000 7f6471ffb640 1 -- 172.21.15.97:0/3586602880 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64640019f0 con 0x7f64740d1100 2024-03-29T04:08:36.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.220+0000 7f647930e640 1 -- 172.21.15.97:0/3586602880 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64740d1100 msgr2=0x7f647409aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.220+0000 7f647930e640 1 --2- 172.21.15.97:0/3586602880 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64740d1100 0x7f647409aa30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f64680097d0 tx=0x7f646802b800 comp rx=0 tx=0).stop 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.220+0000 7f647930e640 1 -- 172.21.15.97:0/3586602880 shutdown_connections 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.220+0000 7f647930e640 1 --2- 172.21.15.97:0/3586602880 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64740d1100 0x7f647409aa30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.220+0000 7f647930e640 1 -- 172.21.15.97:0/3586602880 >> 172.21.15.97:0/3586602880 conn(0x7f64740b3f00 msgr2=0x7f64740b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.220+0000 7f647930e640 1 -- 172.21.15.97:0/3586602880 shutdown_connections 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.220+0000 7f647930e640 1 -- 172.21.15.97:0/3586602880 wait complete. 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.221+0000 7f647930e640 1 Processor -- start 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.221+0000 7f647930e640 1 -- start start 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.222+0000 7f647930e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64740ce120 0x7f6474156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.222+0000 7f647930e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f647415c790 0x7f64740cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.222+0000 7f647930e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64740b7180 0x7f64740af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.222+0000 7f647930e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f64740cd920 con 0x7f647415c790 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.222+0000 7f647930e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f64740cdc90 con 0x7f64740b7180 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.222+0000 7f647930e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f647409af70 con 0x7f64740ce120 2024-03-29T04:08:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.222+0000 7f64727fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f647415c790 0x7f64740cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.222+0000 7f64727fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f647415c790 0x7f64740cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51366/0 (socket says 172.21.15.97:51366) 2024-03-29T04:08:36.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.222+0000 7f64727fc640 1 -- 172.21.15.97:0/894159695 learned_addr learned my addr 172.21.15.97:0/894159695 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:36.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.223+0000 7f64737fe640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64740b7180 0x7f64740af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.223+0000 7f6472ffd640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64740ce120 0x7f6474156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.223+0000 7f64727fc640 1 -- 172.21.15.97:0/894159695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64740ce120 msgr2=0x7f6474156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.223+0000 7f64727fc640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64740ce120 0x7f6474156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.223+0000 7f64727fc640 1 -- 172.21.15.97:0/894159695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64740b7180 msgr2=0x7f64740af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.223+0000 7f64727fc640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64740b7180 0x7f64740af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.223+0000 7f64727fc640 1 -- 172.21.15.97:0/894159695 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64740b60e0 con 0x7f647415c790 2024-03-29T04:08:36.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.223+0000 7f64727fc640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f647415c790 0x7f64740cc160 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f646802b840 tx=0x7f64680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:36.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.223+0000 7f645b7fe640 1 -- 172.21.15.97:0/894159695 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468040020 con 0x7f647415c790 2024-03-29T04:08:36.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.223+0000 7f645b7fe640 1 -- 172.21.15.97:0/894159695 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6468038040 con 0x7f647415c790 2024-03-29T04:08:36.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.223+0000 7f645b7fe640 1 -- 172.21.15.97:0/894159695 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468037070 con 0x7f647415c790 2024-03-29T04:08:36.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.223+0000 7f647930e640 1 -- 172.21.15.97:0/894159695 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64740b63c0 con 0x7f647415c790 2024-03-29T04:08:36.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.223+0000 7f647930e640 1 -- 172.21.15.97:0/894159695 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64740b6980 con 0x7f647415c790 2024-03-29T04:08:36.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.224+0000 7f647930e640 1 -- 172.21.15.97:0/894159695 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6474068d50 con 0x7f647415c790 2024-03-29T04:08:36.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.224+0000 7f645b7fe640 1 -- 172.21.15.97:0/894159695 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6468043020 con 0x7f647415c790 2024-03-29T04:08:36.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.225+0000 7f645b7fe640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f646006c530 0x7f646006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:36.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.225+0000 7f6472ffd640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f646006c530 0x7f646006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:36.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.225+0000 7f645b7fe640 1 -- 172.21.15.97:0/894159695 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f64680acbb0 con 0x7f647415c790 2024-03-29T04:08:36.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.225+0000 7f6472ffd640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f646006c530 0x7f646006e9f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f646c0011e0 tx=0x7f646c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.229+0000 7f645b7fe640 1 -- 172.21.15.97:0/894159695 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f646804a030 con 0x7f647415c790 2024-03-29T04:08:36.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.449+0000 7f647930e640 1 -- 172.21.15.97:0/894159695 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f64740b4ab0 con 0x7f647415c790 2024-03-29T04:08:36.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.449+0000 7f645b7fe640 1 -- 172.21.15.97:0/894159695 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f646807b1d0 con 0x7f647415c790 2024-03-29T04:08:36.429 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T04:08:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.451+0000 7f647930e640 1 -- 172.21.15.97:0/894159695 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f646006c530 msgr2=0x7f646006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.451+0000 7f647930e640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f646006c530 0x7f646006e9f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f646c0011e0 tx=0x7f646c009040 comp rx=0 tx=0).stop 2024-03-29T04:08:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.451+0000 7f647930e640 1 -- 172.21.15.97:0/894159695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f647415c790 msgr2=0x7f64740cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.451+0000 7f647930e640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f647415c790 0x7f64740cc160 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f646802b840 tx=0x7f64680046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.451+0000 7f647930e640 1 -- 172.21.15.97:0/894159695 shutdown_connections 2024-03-29T04:08:36.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.451+0000 7f647930e640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f646006c530 0x7f646006e9f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.451+0000 7f647930e640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64740b7180 0x7f64740af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.451+0000 7f647930e640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f647415c790 0x7f64740cc160 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.451+0000 7f647930e640 1 --2- 172.21.15.97:0/894159695 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64740ce120 0x7f6474156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:36.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.451+0000 7f647930e640 1 -- 172.21.15.97:0/894159695 >> 172.21.15.97:0/894159695 conn(0x7f64740b3f00 msgr2=0x7f647406faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:36.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.451+0000 7f647930e640 1 -- 172.21.15.97:0/894159695 shutdown_connections 2024-03-29T04:08:36.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36.451+0000 7f647930e640 1 -- 172.21.15.97:0/894159695 wait complete. 2024-03-29T04:08:36.446 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-29T04:08:36.446 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:08:36.446 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-03-29T04:08:36.446 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T04:08:36.446 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:08:36.447 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-29T04:08:36.447 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-29T04:08:36.447 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:08:36.447 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:08:36.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:36.448 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-03-29T04:08:36.464 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:08:36.464 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:08:36.464 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:08:36.464 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:36.464 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-03-29T04:08:36.464 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:36.464 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:08:36.464 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:36.464 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42653sec preferred_lft 42653sec 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:4d:74:f4:86:4e brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:08:36.465 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:36.466 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::dccd:95ff:fefa:4739/64 scope link 2024-03-29T04:08:36.466 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:36.466 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:08:36.466 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:4d:74:f4:86:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:08:36.466 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c4d:74ff:fef4:864e/64 scope link 2024-03-29T04:08:36.466 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:36.466 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:08:36.466 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether de:56:55:88:30:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:08:36.466 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::dc56:55ff:fe88:3049/64 scope link 2024-03-29T04:08:36.466 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:36.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:36.467 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:08:36.531 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-03-29T04:08:36.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:36.532 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-03-29T04:08:36.588 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-03-29T04:08:36.588 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:08:36.588 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:08:36.588 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:08:36.588 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:08:36.589 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:08:36.589 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-03-29T04:08:36.589 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:08:36.589 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:08:36.589 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:08:36.589 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-03-29T04:08:36.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:08:36.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:36.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:36.726 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-29T04:08:36.742 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T04:08:36.742 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T04:08:36.742 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:08:36.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:36.742 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-03-29T04:08:36.809 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:08:36.810 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T04:08:36.865 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:08:36.866 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:08:36.866 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:08:36.920 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-29T04:08:36.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:36.920 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:08:37.004 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:08:37.004 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:08:37.011 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:08:37.012 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:08:37.012 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.154:3300,172.21.15.154:3301" 2024-03-29T04:08:37.012 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f.cephfs=/ 2024-03-29T04:08:37.012 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.154:3300/172.21.15.154:3301" will pass to kernel 2024-03-29T04:08:37.012 INFO:teuthology.orchestra.run.smithi097.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-29T04:08:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: You just mounted a file system that supports labels which does not 2024-03-29T04:08:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-29T04:08:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: applications will generate AVC messages and not be allowed access to 2024-03-29T04:08:37.012 INFO:teuthology.orchestra.run.smithi097.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-29T04:08:37.014 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-29T04:08:37.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:37.014 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:08:37.081 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:08:37.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:08:37.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:37.182 INFO:teuthology.orchestra.run.smithi097.stdout:client4907 2024-03-29T04:08:37.182 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:08:37.182 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:37.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:08:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.360+0000 7fa08b57c640 1 Processor -- start 2024-03-29T04:08:37.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.360+0000 7fa08b57c640 1 -- start start 2024-03-29T04:08:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.361+0000 7fa08b57c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa08415e930 0x7fa0840cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:37.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.361+0000 7fa08b57c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa0840cf670 0x7fa0840d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.361+0000 7fa08b57c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0840cfb30 con 0x7fa0840b3c50 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.361+0000 7fa08b57c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fa08406c780 con 0x7fa0840cf670 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.361+0000 7fa08b57c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa0840cfe10 con 0x7fa08415e930 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.361+0000 7fa089b19640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa0840b3c50 0x7fa0840d0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55846/0 (socket says 172.21.15.97:55846) 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.361+0000 7fa089b19640 1 -- 172.21.15.97:0/2145923996 learned_addr learned my addr 172.21.15.97:0/2145923996 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.361+0000 7fa088b17640 1 --2- 172.21.15.97:0/2145923996 >> v2:172.21.15.154:3300/0 conn(0x7fa0840cf670 0x7fa0840d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.361+0000 7fa089318640 1 --2- 172.21.15.97:0/2145923996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa08415e930 0x7fa0840cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.362+0000 7fa07bfff640 1 -- 172.21.15.97:0/2145923996 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1620074043 0 0) 0x7fa0840cfb30 con 0x7fa0840b3c50 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.362+0000 7fa07bfff640 1 -- 172.21.15.97:0/2145923996 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa070003520 con 0x7fa0840b3c50 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.362+0000 7fa089318640 1 -- 172.21.15.97:0/2145923996 >> v2:172.21.15.154:3300/0 conn(0x7fa0840cf670 msgr2=0x7fa0840d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.362+0000 7fa089318640 1 --2- 172.21.15.97:0/2145923996 >> v2:172.21.15.154:3300/0 conn(0x7fa0840cf670 0x7fa0840d2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.362+0000 7fa089318640 1 -- 172.21.15.97:0/2145923996 >> v1:172.21.15.97:6789/0 conn(0x7fa0840b3c50 legacy=0x7fa0840d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.362+0000 7fa089318640 1 -- 172.21.15.97:0/2145923996 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0840d00f0 con 0x7fa08415e930 2024-03-29T04:08:37.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.362+0000 7fa088b17640 1 --2- 172.21.15.97:0/2145923996 >> v2:172.21.15.154:3300/0 conn(0x7fa0840cf670 0x7fa0840d2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:37.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.363+0000 7fa089318640 1 --2- 172.21.15.97:0/2145923996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa08415e930 0x7fa0840cf130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa0800097a0 tx=0x7fa08002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54c0e7fc367aa206 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:37.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.363+0000 7fa07bfff640 1 -- 172.21.15.97:0/2145923996 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0800315a0 con 0x7fa08415e930 2024-03-29T04:08:37.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.363+0000 7fa07bfff640 1 -- 172.21.15.97:0/2145923996 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa080037070 con 0x7fa08415e930 2024-03-29T04:08:37.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.363+0000 7fa07bfff640 1 -- 172.21.15.97:0/2145923996 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080031940 con 0x7fa08415e930 2024-03-29T04:08:37.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.363+0000 7fa08b57c640 1 -- 172.21.15.97:0/2145923996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa08415e930 msgr2=0x7fa0840cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:37.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.363+0000 7fa08b57c640 1 --2- 172.21.15.97:0/2145923996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa08415e930 0x7fa0840cf130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa0800097a0 tx=0x7fa08002b840 comp rx=0 tx=0).stop 2024-03-29T04:08:37.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.363+0000 7fa08b57c640 1 -- 172.21.15.97:0/2145923996 shutdown_connections 2024-03-29T04:08:37.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.363+0000 7fa08b57c640 1 --2- 172.21.15.97:0/2145923996 >> v2:172.21.15.154:3300/0 conn(0x7fa0840cf670 0x7fa0840d2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:37.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.363+0000 7fa08b57c640 1 --2- 172.21.15.97:0/2145923996 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa08415e930 0x7fa0840cf130 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:37.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.363+0000 7fa08b57c640 1 -- 172.21.15.97:0/2145923996 >> 172.21.15.97:0/2145923996 conn(0x7fa08408bfc0 msgr2=0x7fa0840d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:37.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.363+0000 7fa08b57c640 1 -- 172.21.15.97:0/2145923996 shutdown_connections 2024-03-29T04:08:37.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.363+0000 7fa08b57c640 1 -- 172.21.15.97:0/2145923996 wait complete. 2024-03-29T04:08:37.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.364+0000 7fa08b57c640 1 Processor -- start 2024-03-29T04:08:37.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.365+0000 7fa08b57c640 1 -- start start 2024-03-29T04:08:37.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.365+0000 7fa08b57c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0840b3c50 0x7fa08415b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:37.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.365+0000 7fa08b57c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0840cf670 0x7fa0840a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:37.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.365+0000 7fa08b57c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa08415e930 0x7fa084158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.365+0000 7fa08b57c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa08415ee90 con 0x7fa08415e930 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.365+0000 7fa08b57c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa08415ed10 con 0x7fa0840cf670 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.365+0000 7fa08b57c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa08406dc20 con 0x7fa0840b3c50 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.365+0000 7fa089b19640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa08415e930 0x7fa084158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.365+0000 7fa089b19640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa08415e930 0x7fa084158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51374/0 (socket says 172.21.15.97:51374) 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.365+0000 7fa089b19640 1 -- 172.21.15.97:0/3967647781 learned_addr learned my addr 172.21.15.97:0/3967647781 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.366+0000 7fa088b17640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0840cf670 0x7fa0840a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.366+0000 7fa089b19640 1 -- 172.21.15.97:0/3967647781 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0840b3c50 msgr2=0x7fa08415b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.366+0000 7fa089318640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0840b3c50 0x7fa08415b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.366+0000 7fa089b19640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0840b3c50 0x7fa08415b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.366+0000 7fa089b19640 1 -- 172.21.15.97:0/3967647781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0840cf670 msgr2=0x7fa0840a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.366+0000 7fa089b19640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0840cf670 0x7fa0840a12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.366+0000 7fa089b19640 1 -- 172.21.15.97:0/3967647781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0840a1da0 con 0x7fa08415e930 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.366+0000 7fa089b19640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa08415e930 0x7fa084158720 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00cae0 tx=0x7fa07c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:37.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.366+0000 7fa079ffb640 1 -- 172.21.15.97:0/3967647781 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c017070 con 0x7fa08415e930 2024-03-29T04:08:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.366+0000 7fa079ffb640 1 -- 172.21.15.97:0/3967647781 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07c00ac40 con 0x7fa08415e930 2024-03-29T04:08:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.366+0000 7fa079ffb640 1 -- 172.21.15.97:0/3967647781 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c012640 con 0x7fa08415e930 2024-03-29T04:08:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.366+0000 7fa08b57c640 1 -- 172.21.15.97:0/3967647781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0840a2080 con 0x7fa08415e930 2024-03-29T04:08:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.366+0000 7fa08b57c640 1 -- 172.21.15.97:0/3967647781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0840cce50 con 0x7fa08415e930 2024-03-29T04:08:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.367+0000 7fa08b57c640 1 -- 172.21.15.97:0/3967647781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa084068640 con 0x7fa08415e930 2024-03-29T04:08:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.367+0000 7fa079ffb640 1 -- 172.21.15.97:0/3967647781 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa07c01a070 con 0x7fa08415e930 2024-03-29T04:08:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.368+0000 7fa079ffb640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa06806c560 0x7fa06806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.368+0000 7fa089318640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa06806c560 0x7fa06806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.368+0000 7fa079ffb640 1 -- 172.21.15.97:0/3967647781 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa07c08b560 con 0x7fa08415e930 2024-03-29T04:08:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.368+0000 7fa089318640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa06806c560 0x7fa06806ea20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa080036000 tx=0x7fa080008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.372+0000 7fa079ffb640 1 -- 172.21.15.97:0/3967647781 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa07c059af0 con 0x7fa08415e930 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.584+0000 7fa08b57c640 1 -- 172.21.15.97:0/3967647781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa0840a2250 con 0x7fa08415e930 2024-03-29T04:08:37.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.585+0000 7fa079ffb640 1 -- 172.21.15.97:0/3967647781 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7fa07c0593f0 con 0x7fa08415e930 2024-03-29T04:08:37.565 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T04:08:37.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.587+0000 7fa08b57c640 1 -- 172.21.15.97:0/3967647781 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa06806c560 msgr2=0x7fa06806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.587+0000 7fa08b57c640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa06806c560 0x7fa06806ea20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa080036000 tx=0x7fa080008030 comp rx=0 tx=0).stop 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.587+0000 7fa08b57c640 1 -- 172.21.15.97:0/3967647781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa08415e930 msgr2=0x7fa084158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.587+0000 7fa08b57c640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa08415e930 0x7fa084158720 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00cae0 tx=0x7fa07c00a060 comp rx=0 tx=0).stop 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.587+0000 7fa08b57c640 1 -- 172.21.15.97:0/3967647781 shutdown_connections 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.587+0000 7fa08b57c640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa06806c560 0x7fa06806ea20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.587+0000 7fa08b57c640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa08415e930 0x7fa084158720 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.587+0000 7fa08b57c640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0840cf670 0x7fa0840a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.587+0000 7fa08b57c640 1 --2- 172.21.15.97:0/3967647781 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0840b3c50 0x7fa08415b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.587+0000 7fa08b57c640 1 -- 172.21.15.97:0/3967647781 >> 172.21.15.97:0/3967647781 conn(0x7fa08408bfc0 msgr2=0x7fa084161a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.587+0000 7fa08b57c640 1 -- 172.21.15.97:0/3967647781 shutdown_connections 2024-03-29T04:08:37.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:37.587+0000 7fa08b57c640 1 -- 172.21.15.97:0/3967647781 wait complete. 2024-03-29T04:08:37.577 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-29T04:08:37.577 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:08:37.577 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-03-29T04:08:37.577 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-29T04:08:37.577 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:08:37.577 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-29T04:08:37.578 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-29T04:08:37.578 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:08:37.578 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:08:37.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:37.578 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-03-29T04:08:37.599 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:08:37.599 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:08:37.599 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:08:37.599 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:37.599 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-03-29T04:08:37.599 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:37.599 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:08:37.600 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:37.600 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-03-29T04:08:37.600 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:08:37.600 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:37.600 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-03-29T04:08:37.600 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:08:37.600 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:37.600 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:08:37.600 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:37.600 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:08:37.600 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42667sec preferred_lft 42667sec 2024-03-29T04:08:37.600 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether d6:7d:69:db:81:04 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::60a9:efff:fe1a:efc0/64 scope link 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether d6:7d:69:db:81:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::d47d:69ff:fedb:8104/64 scope link 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether fe:91:7a:f2:67:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:08:37.601 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::fc91:7aff:fef2:67af/64 scope link 2024-03-29T04:08:37.602 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:08:37.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:37.602 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:08:37.669 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-03-29T04:08:37.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:37.671 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-03-29T04:08:37.729 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-03-29T04:08:37.729 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:08:37.729 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:08:37.729 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:08:37.729 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:08:37.730 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:08:37.730 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-29T04:08:37.730 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:08:37.730 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:08:37.730 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:08:37.730 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-29T04:08:37.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:08:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:37.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:37.870 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-29T04:08:37.885 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-29T04:08:37.885 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T04:08:37.886 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:08:37.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:37.886 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2024-03-29T04:08:37.953 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:08:37.953 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-29T04:08:38.008 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:08:38.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:08:38.009 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:08:38.064 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-29T04:08:38.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:38.064 DEBUG:teuthology.orchestra.run.smithi154:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:08:38.149 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:08:38.149 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:08:38.155 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:08:38.156 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:08:38.156 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.154:3300,172.21.15.154:3301" 2024-03-29T04:08:38.156 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: 1@51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f.cephfs=/ 2024-03-29T04:08:38.156 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.154:3300/172.21.15.154:3301" will pass to kernel 2024-03-29T04:08:38.158 INFO:teuthology.orchestra.run.smithi154.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-29T04:08:38.158 INFO:teuthology.orchestra.run.smithi154.stdout: You just mounted a file system that supports labels which does not 2024-03-29T04:08:38.158 INFO:teuthology.orchestra.run.smithi154.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-29T04:08:38.159 INFO:teuthology.orchestra.run.smithi154.stdout: applications will generate AVC messages and not be allowed access to 2024-03-29T04:08:38.159 INFO:teuthology.orchestra.run.smithi154.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-29T04:08:38.159 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-29T04:08:38.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:38.159 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:08:38.192 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:08:38.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:08:38.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-29T04:08:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:08:38.288 INFO:teuthology.orchestra.run.smithi154.stdout:client5280 2024-03-29T04:08:38.288 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:08:38.288 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:08:38.303 DEBUG:tasks.cephfs.test_uninlining:setting inline_data:1 2024-03-29T04:08:38.304 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs inline_data 1 --yes-i-really-really-mean-it 2024-03-29T04:08:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.439+0000 7fcbba0a0640 1 Processor -- start 2024-03-29T04:08:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.439+0000 7fcbba0a0640 1 -- start start 2024-03-29T04:08:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.440+0000 7fcbba0a0640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcbb40d4f20 0x7fcbb4162ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.440+0000 7fcbba0a0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbb40b37c0 con 0x7fcbb415e780 2024-03-29T04:08:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.440+0000 7fcbba0a0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbb40d53e0 con 0x7fcbb415eb60 2024-03-29T04:08:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.440+0000 7fcbba0a0640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fcbb4163430 con 0x7fcbb40d4f20 2024-03-29T04:08:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.440+0000 7fcbaaffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcbb40d4f20 0x7fcbb4162ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.440+0000 7fcbb37fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcbb415e780 0x7fcbb40b1170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55858/0 (socket says 172.21.15.97:55858) 2024-03-29T04:08:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.440+0000 7fcbaaffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcbb40d4f20 0x7fcbb4162ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51390/0 (socket says 172.21.15.97:51390) 2024-03-29T04:08:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.440+0000 7fcbb37fe640 1 -- 172.21.15.97:0/3492463255 learned_addr learned my addr 172.21.15.97:0/3492463255 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.440+0000 7fcbaaffd640 1 -- 172.21.15.97:0/3492463255 >> v1:172.21.15.154:6789/0 conn(0x7fcbb415eb60 legacy=0x7fcbb40d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.440+0000 7fcbaaffd640 1 -- 172.21.15.97:0/3492463255 >> v1:172.21.15.97:6789/0 conn(0x7fcbb415e780 legacy=0x7fcbb40b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.440+0000 7fcbaaffd640 1 -- 172.21.15.97:0/3492463255 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb40cf790 con 0x7fcbb40d4f20 2024-03-29T04:08:38.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.441+0000 7fcbaaffd640 1 --2- 172.21.15.97:0/3492463255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb40d4f20 0x7fcbb4162ef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009790 tx=0x7fcb9c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cda212856411c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:38.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.441+0000 7fcbb2ffd640 1 -- 172.21.15.97:0/3492463255 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c034070 con 0x7fcbb40d4f20 2024-03-29T04:08:38.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.441+0000 7fcbb2ffd640 1 -- 172.21.15.97:0/3492463255 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb9c030610 con 0x7fcbb40d4f20 2024-03-29T04:08:38.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.441+0000 7fcbb2ffd640 1 -- 172.21.15.97:0/3492463255 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0309e0 con 0x7fcbb40d4f20 2024-03-29T04:08:38.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.442+0000 7fcbba0a0640 1 -- 172.21.15.97:0/3492463255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb40d4f20 msgr2=0x7fcbb4162ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:38.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.442+0000 7fcbba0a0640 1 --2- 172.21.15.97:0/3492463255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb40d4f20 0x7fcbb4162ef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009790 tx=0x7fcb9c02b960 comp rx=0 tx=0).stop 2024-03-29T04:08:38.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.442+0000 7fcbba0a0640 1 -- 172.21.15.97:0/3492463255 shutdown_connections 2024-03-29T04:08:38.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.442+0000 7fcbba0a0640 1 --2- 172.21.15.97:0/3492463255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb40d4f20 0x7fcbb4162ef0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:38.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.442+0000 7fcbba0a0640 1 -- 172.21.15.97:0/3492463255 >> 172.21.15.97:0/3492463255 conn(0x7fcbb408bfc0 msgr2=0x7fcbb40d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:38.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.442+0000 7fcbba0a0640 1 -- 172.21.15.97:0/3492463255 shutdown_connections 2024-03-29T04:08:38.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.442+0000 7fcbba0a0640 1 -- 172.21.15.97:0/3492463255 wait complete. 2024-03-29T04:08:38.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.443+0000 7fcbba0a0640 1 Processor -- start 2024-03-29T04:08:38.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.443+0000 7fcbba0a0640 1 -- start start 2024-03-29T04:08:38.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbba0a0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbb40d4f20 0x7fcbb4161650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:38.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbba0a0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb415e780 0x7fcbb41f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:38.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbba0a0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbb415eb60 0x7fcbb40d2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:38.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbba0a0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb40cf440 con 0x7fcbb415e780 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbba0a0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb40cfde0 con 0x7fcbb415eb60 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbba0a0640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb40b1880 con 0x7fcbb40d4f20 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbaaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb415e780 0x7fcbb41f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbaaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb415e780 0x7fcbb41f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51396/0 (socket says 172.21.15.97:51396) 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbaaffd640 1 -- 172.21.15.97:0/56195792 learned_addr learned my addr 172.21.15.97:0/56195792 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbb37fe640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbb40d4f20 0x7fcbb4161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbb3fff640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbb415eb60 0x7fcbb40d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbaaffd640 1 -- 172.21.15.97:0/56195792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbb40d4f20 msgr2=0x7fcbb4161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbaaffd640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbb40d4f20 0x7fcbb4161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbaaffd640 1 -- 172.21.15.97:0/56195792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbb415eb60 msgr2=0x7fcbb40d2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbaaffd640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbb415eb60 0x7fcbb40d2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbaaffd640 1 -- 172.21.15.97:0/56195792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb4160bb0 con 0x7fcbb415e780 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.444+0000 7fcbb3fff640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbb415eb60 0x7fcbb40d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:38.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.445+0000 7fcbaaffd640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb415e780 0x7fcbb41f4380 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c02bf70 tx=0x7fcb9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:38.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.445+0000 7fcbb0ff9640 1 -- 172.21.15.97:0/56195792 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c034040 con 0x7fcbb415e780 2024-03-29T04:08:38.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.445+0000 7fcbb0ff9640 1 -- 172.21.15.97:0/56195792 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb9c03acc0 con 0x7fcbb415e780 2024-03-29T04:08:38.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.445+0000 7fcbba0a0640 1 -- 172.21.15.97:0/56195792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb41f4dd0 con 0x7fcbb415e780 2024-03-29T04:08:38.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.445+0000 7fcbb0ff9640 1 -- 172.21.15.97:0/56195792 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c044630 con 0x7fcbb415e780 2024-03-29T04:08:38.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.445+0000 7fcbba0a0640 1 -- 172.21.15.97:0/56195792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb4160e50 con 0x7fcbb415e780 2024-03-29T04:08:38.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.446+0000 7fcbba0a0640 1 -- 172.21.15.97:0/56195792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb4068640 con 0x7fcbb415e780 2024-03-29T04:08:38.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.446+0000 7fcbb0ff9640 1 -- 172.21.15.97:0/56195792 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcb9c04b060 con 0x7fcbb415e780 2024-03-29T04:08:38.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.447+0000 7fcbb0ff9640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcb9406c530 0x7fcb9406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:38.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.447+0000 7fcbb0ff9640 1 -- 172.21.15.97:0/56195792 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fcb9c0ad5b0 con 0x7fcbb415e780 2024-03-29T04:08:38.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.447+0000 7fcbb37fe640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcb9406c530 0x7fcb9406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:38.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.447+0000 7fcbb37fe640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcb9406c530 0x7fcb9406e9f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcba4005880 tx=0x7fcba4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:38.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.451+0000 7fcbb0ff9640 1 -- 172.21.15.97:0/56195792 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcb9c03f070 con 0x7fcbb415e780 2024-03-29T04:08:38.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:38.706+0000 7fcbba0a0640 1 -- 172.21.15.97:0/56195792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "1", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcbb41f52f0 con 0x7fcbb415e780 2024-03-29T04:08:39.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:39.256+0000 7fcbb0ff9640 1 -- 172.21.15.97:0/56195792 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "1", "yes_i_really_really_mean_it": true}]=0 inline data enabled v15) ==== 165+0+0 (secure 0 0 0) 0x7fcb9c04b350 con 0x7fcbb415e780 2024-03-29T04:08:39.236 INFO:teuthology.orchestra.run.smithi097.stderr:inline data enabled 2024-03-29T04:08:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:39.258+0000 7fcbba0a0640 1 -- 172.21.15.97:0/56195792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcb9406c530 msgr2=0x7fcb9406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:39.258+0000 7fcbba0a0640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcb9406c530 0x7fcb9406e9f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcba4005880 tx=0x7fcba4009040 comp rx=0 tx=0).stop 2024-03-29T04:08:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:39.258+0000 7fcbba0a0640 1 -- 172.21.15.97:0/56195792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb415e780 msgr2=0x7fcbb41f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:39.258+0000 7fcbba0a0640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb415e780 0x7fcbb41f4380 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c02bf70 tx=0x7fcb9c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:39.258+0000 7fcbba0a0640 1 -- 172.21.15.97:0/56195792 shutdown_connections 2024-03-29T04:08:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:39.258+0000 7fcbba0a0640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fcb9406c530 0x7fcb9406e9f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:39.258+0000 7fcbba0a0640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbb415eb60 0x7fcbb40d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:39.258+0000 7fcbba0a0640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbb415e780 0x7fcbb41f4380 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:39.258+0000 7fcbba0a0640 1 --2- 172.21.15.97:0/56195792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbb40d4f20 0x7fcbb4161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:39.258+0000 7fcbba0a0640 1 -- 172.21.15.97:0/56195792 >> 172.21.15.97:0/56195792 conn(0x7fcbb408bfc0 msgr2=0x7fcbb406df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:39.258+0000 7fcbba0a0640 1 -- 172.21.15.97:0/56195792 shutdown_connections 2024-03-29T04:08:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:39.258+0000 7fcbba0a0640 1 -- 172.21.15.97:0/56195792 wait complete. 2024-03-29T04:08:39.249 DEBUG:tasks.cephfs.test_uninlining:fetching tarball 2024-03-29T04:08:39.250 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && wget http://download.ceph.com/qa/linux-5.4.tar.gz 2024-03-29T04:08:39.285 INFO:teuthology.orchestra.run.smithi097.stderr:--2024-03-29 04:08:39-- http://download.ceph.com/qa/linux-5.4.tar.gz 2024-03-29T04:08:39.285 INFO:teuthology.orchestra.run.smithi097.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-29T04:08:39.323 INFO:teuthology.orchestra.run.smithi097.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-29T04:08:39.360 INFO:teuthology.orchestra.run.smithi097.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-29T04:08:39.360 INFO:teuthology.orchestra.run.smithi097.stderr:Length: 172616875 (165M) [application/octet-stream] 2024-03-29T04:08:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:Saving to: ‘linux-5.4.tar.gz’ 2024-03-29T04:08:39.365 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-29T04:08:39.432 INFO:teuthology.orchestra.run.smithi097.stderr: 0K .......... .......... .......... .......... .......... 0% 734K 3m49s 2024-03-29T04:08:39.469 INFO:teuthology.orchestra.run.smithi097.stderr: 50K .......... .......... .......... .......... .......... 0% 1.32M 2m57s 2024-03-29T04:08:39.508 INFO:teuthology.orchestra.run.smithi097.stderr: 100K .......... .......... .......... .......... .......... 0% 1.37M 2m38s 2024-03-29T04:08:39.508 INFO:teuthology.orchestra.run.smithi097.stderr: 150K .......... .......... .......... .......... .......... 0% 112M 1m59s 2024-03-29T04:08:39.547 INFO:teuthology.orchestra.run.smithi097.stderr: 200K .......... .......... .......... .......... .......... 0% 1.39M 1m59s 2024-03-29T04:08:39.547 INFO:teuthology.orchestra.run.smithi097.stderr: 250K .......... .......... .......... .......... .......... 0% 78.8M 99s 2024-03-29T04:08:39.547 INFO:teuthology.orchestra.run.smithi097.stderr: 300K .......... .......... .......... .......... .......... 0% 110M 85s 2024-03-29T04:08:39.547 INFO:teuthology.orchestra.run.smithi097.stderr: 350K .......... .......... .......... .......... .......... 0% 105M 75s 2024-03-29T04:08:39.592 INFO:teuthology.orchestra.run.smithi097.stderr: 400K .......... .......... .......... .......... .......... 0% 1.43M 79s 2024-03-29T04:08:39.593 INFO:teuthology.orchestra.run.smithi097.stderr: 450K .......... .......... .......... .......... .......... 0% 75.9M 71s 2024-03-29T04:08:39.593 INFO:teuthology.orchestra.run.smithi097.stderr: 500K .......... .......... .......... .......... .......... 0% 106M 65s 2024-03-29T04:08:39.593 INFO:teuthology.orchestra.run.smithi097.stderr: 550K .......... .......... .......... .......... .......... 0% 113M 60s 2024-03-29T04:08:39.593 INFO:teuthology.orchestra.run.smithi097.stderr: 600K .......... .......... .......... .......... .......... 0% 58.2M 55s 2024-03-29T04:08:39.593 INFO:teuthology.orchestra.run.smithi097.stderr: 650K .......... .......... .......... .......... .......... 0% 89.2M 52s 2024-03-29T04:08:39.593 INFO:teuthology.orchestra.run.smithi097.stderr: 700K .......... .......... .......... .......... .......... 0% 113M 48s 2024-03-29T04:08:39.593 INFO:teuthology.orchestra.run.smithi097.stderr: 750K .......... .......... .......... .......... .......... 0% 33.2M 45s 2024-03-29T04:08:39.783 INFO:teuthology.orchestra.run.smithi097.stderr: 800K .......... .......... .......... .......... .......... 0% 1.59M 49s 2024-03-29T04:08:39.783 INFO:teuthology.orchestra.run.smithi097.stderr: 850K .......... .......... .......... .......... .......... 0% 113M 46s 2024-03-29T04:08:39.783 INFO:teuthology.orchestra.run.smithi097.stderr: 900K .......... .......... .......... .......... .......... 0% 59.3M 44s 2024-03-29T04:08:39.783 INFO:teuthology.orchestra.run.smithi097.stderr: 950K .......... .......... .......... .......... .......... 0% 103M 42s 2024-03-29T04:08:39.783 INFO:teuthology.orchestra.run.smithi097.stderr: 1000K .......... .......... .......... .......... .......... 0% 115M 40s 2024-03-29T04:08:39.783 INFO:teuthology.orchestra.run.smithi097.stderr: 1050K .......... .......... .......... .......... .......... 0% 68.6M 38s 2024-03-29T04:08:39.783 INFO:teuthology.orchestra.run.smithi097.stderr: 1100K .......... .......... .......... .......... .......... 0% 88.0M 37s 2024-03-29T04:08:39.783 INFO:teuthology.orchestra.run.smithi097.stderr: 1150K .......... .......... .......... .......... .......... 0% 57.6M 35s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1200K .......... .......... .......... .......... .......... 0% 35.0M 34s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1250K .......... .......... .......... .......... .......... 0% 29.7M 33s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1300K .......... .......... .......... .......... .......... 0% 54.4M 32s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1350K .......... .......... .......... .......... .......... 0% 31.8M 31s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1400K .......... .......... .......... .......... .......... 0% 29.8M 30s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1450K .......... .......... .......... .......... .......... 0% 18.9M 29s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1500K .......... .......... .......... .......... .......... 0% 10.4M 29s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1550K .......... .......... .......... .......... .......... 0% 29.3M 28s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1600K .......... .......... .......... .......... .......... 0% 3.24M 29s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1650K .......... .......... .......... .......... .......... 1% 74.5M 28s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1700K .......... .......... .......... .......... .......... 1% 76.2M 27s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1750K .......... .......... .......... .......... .......... 1% 67.6M 26s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1800K .......... .......... .......... .......... .......... 1% 89.1M 26s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1850K .......... .......... .......... .......... .......... 1% 56.2M 25s 2024-03-29T04:08:39.784 INFO:teuthology.orchestra.run.smithi097.stderr: 1900K .......... .......... .......... .......... .......... 1% 74.7M 25s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 1950K .......... .......... .......... .......... .......... 1% 52.8M 24s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 2000K .......... .......... .......... .......... .......... 1% 19.0M 24s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 2050K .......... .......... .......... .......... .......... 1% 46.6M 23s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 2100K .......... .......... .......... .......... .......... 1% 19.5M 23s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 2150K .......... .......... .......... .......... .......... 1% 23.4M 22s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 2200K .......... .......... .......... .......... .......... 1% 44.6M 22s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 2250K .......... .......... .......... .......... .......... 1% 15.5M 22s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 2300K .......... .......... .......... .......... .......... 1% 28.4M 21s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 2350K .......... .......... .......... .......... .......... 1% 15.6M 21s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 2400K .......... .......... .......... .......... .......... 1% 34.5M 21s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 2450K .......... .......... .......... .......... .......... 1% 21.2M 21s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 2500K .......... .......... .......... .......... .......... 1% 24.3M 20s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 2550K .......... .......... .......... .......... .......... 1% 47.7M 20s 2024-03-29T04:08:39.785 INFO:teuthology.orchestra.run.smithi097.stderr: 2600K .......... .......... .......... .......... .......... 1% 28.9M 20s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 2650K .......... .......... .......... .......... .......... 1% 35.3M 19s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 2700K .......... .......... .......... .......... .......... 1% 20.7M 19s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 2750K .......... .......... .......... .......... .......... 1% 23.9M 19s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 2800K .......... .......... .......... .......... .......... 1% 23.7M 19s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 2850K .......... .......... .......... .......... .......... 1% 27.2M 18s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 2900K .......... .......... .......... .......... .......... 1% 31.1M 18s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 2950K .......... .......... .......... .......... .......... 1% 21.8M 18s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 3000K .......... .......... .......... .......... .......... 1% 38.3M 18s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 3050K .......... .......... .......... .......... .......... 1% 64.3M 18s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 3100K .......... .......... .......... .......... .......... 1% 43.1M 17s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 3150K .......... .......... .......... .......... .......... 1% 38.8M 17s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 3200K .......... .......... .......... .......... .......... 1% 43.1M 17s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 3250K .......... .......... .......... .......... .......... 1% 44.8M 17s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 3300K .......... .......... .......... .......... .......... 1% 49.9M 17s 2024-03-29T04:08:39.786 INFO:teuthology.orchestra.run.smithi097.stderr: 3350K .......... .......... .......... .......... .......... 2% 43.4M 16s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 3400K .......... .......... .......... .......... .......... 2% 2.06M 17s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 3450K .......... .......... .......... .......... .......... 2% 111M 17s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 3500K .......... .......... .......... .......... .......... 2% 108M 17s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 3550K .......... .......... .......... .......... .......... 2% 112M 17s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 3600K .......... .......... .......... .......... .......... 2% 114M 16s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 3650K .......... .......... .......... .......... .......... 2% 1.63M 17s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 3700K .......... .......... .......... .......... .......... 2% 126M 17s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 3750K .......... .......... .......... .......... .......... 2% 115M 17s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 3800K .......... .......... .......... .......... .......... 2% 101M 17s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 3850K .......... .......... .......... .......... .......... 2% 126M 17s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 3900K .......... .......... .......... .......... .......... 2% 112M 16s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 3950K .......... .......... .......... .......... .......... 2% 26.8M 16s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 4000K .......... .......... .......... .......... .......... 2% 113M 16s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 4050K .......... .......... .......... .......... .......... 2% 113M 16s 2024-03-29T04:08:39.787 INFO:teuthology.orchestra.run.smithi097.stderr: 4100K .......... .......... .......... .......... .......... 2% 48.4M 16s 2024-03-29T04:08:39.788 INFO:teuthology.orchestra.run.smithi097.stderr: 4150K .......... .......... .......... .......... .......... 2% 33.4M 16s 2024-03-29T04:08:39.809 INFO:teuthology.orchestra.run.smithi097.stderr: 4200K .......... .......... .......... .......... .......... 2% 1.16M 17s 2024-03-29T04:08:39.809 INFO:teuthology.orchestra.run.smithi097.stderr: 4250K .......... .......... .......... .......... .......... 2% 115M 17s 2024-03-29T04:08:39.825 INFO:teuthology.orchestra.run.smithi097.stderr: 4300K .......... .......... .......... .......... .......... 2% 3.03M 17s 2024-03-29T04:08:39.825 INFO:teuthology.orchestra.run.smithi097.stderr: 4350K .......... .......... .......... .......... .......... 2% 112M 17s 2024-03-29T04:08:39.825 INFO:teuthology.orchestra.run.smithi097.stderr: 4400K .......... .......... .......... .......... .......... 2% 114M 17s 2024-03-29T04:08:39.843 INFO:teuthology.orchestra.run.smithi097.stderr: 4450K .......... .......... .......... .......... .......... 2% 5.71M 17s 2024-03-29T04:08:39.843 INFO:teuthology.orchestra.run.smithi097.stderr: 4500K .......... .......... .......... .......... .......... 2% 112M 17s 2024-03-29T04:08:39.843 INFO:teuthology.orchestra.run.smithi097.stderr: 4550K .......... .......... .......... .......... .......... 2% 111M 17s 2024-03-29T04:08:39.843 INFO:teuthology.orchestra.run.smithi097.stderr: 4600K .......... .......... .......... .......... .......... 2% 113M 16s 2024-03-29T04:08:39.843 INFO:teuthology.orchestra.run.smithi097.stderr: 4650K .......... .......... .......... .......... .......... 2% 112M 16s 2024-03-29T04:08:39.860 INFO:teuthology.orchestra.run.smithi097.stderr: 4700K .......... .......... .......... .......... .......... 2% 2.93M 17s 2024-03-29T04:08:39.860 INFO:teuthology.orchestra.run.smithi097.stderr: 4750K .......... .......... .......... .......... .......... 2% 115M 17s 2024-03-29T04:08:39.860 INFO:teuthology.orchestra.run.smithi097.stderr: 4800K .......... .......... .......... .......... .......... 2% 112M 16s 2024-03-29T04:08:39.860 INFO:teuthology.orchestra.run.smithi097.stderr: 4850K .......... .......... .......... .......... .......... 2% 107M 16s 2024-03-29T04:08:39.860 INFO:teuthology.orchestra.run.smithi097.stderr: 4900K .......... .......... .......... .......... .......... 2% 110M 16s 2024-03-29T04:08:39.885 INFO:teuthology.orchestra.run.smithi097.stderr: 4950K .......... .......... .......... .......... .......... 2% 1.97M 17s 2024-03-29T04:08:39.885 INFO:teuthology.orchestra.run.smithi097.stderr: 5000K .......... .......... .......... .......... .......... 2% 110M 17s 2024-03-29T04:08:39.885 INFO:teuthology.orchestra.run.smithi097.stderr: 5050K .......... .......... .......... .......... .......... 3% 110M 16s 2024-03-29T04:08:39.885 INFO:teuthology.orchestra.run.smithi097.stderr: 5100K .......... .......... .......... .......... .......... 3% 110M 16s 2024-03-29T04:08:39.885 INFO:teuthology.orchestra.run.smithi097.stderr: 5150K .......... .......... .......... .......... .......... 3% 117M 16s 2024-03-29T04:08:40.291 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:08:40.622 INFO:teuthology.orchestra.run.smithi097.stderr: 5200K .......... .......... .......... .......... .......... 3% 3.50M 16s 2024-03-29T04:08:40.623 INFO:teuthology.orchestra.run.smithi097.stderr: 5250K .......... .......... .......... .......... .......... 3% 122M 16s 2024-03-29T04:08:40.623 INFO:teuthology.orchestra.run.smithi097.stderr: 5300K .......... .......... .......... .......... .......... 3% 110M 16s 2024-03-29T04:08:40.623 INFO:teuthology.orchestra.run.smithi097.stderr: 5350K .......... .......... .......... .......... .......... 3% 110M 16s 2024-03-29T04:08:40.623 INFO:teuthology.orchestra.run.smithi097.stderr: 5400K .......... .......... .......... .......... .......... 3% 116M 16s 2024-03-29T04:08:40.623 INFO:teuthology.orchestra.run.smithi097.stderr: 5450K .......... .......... .......... .......... .......... 3% 44.7M 16s 2024-03-29T04:08:40.623 INFO:teuthology.orchestra.run.smithi097.stderr: 5500K .......... .......... .......... .......... .......... 3% 113M 16s 2024-03-29T04:08:40.623 INFO:teuthology.orchestra.run.smithi097.stderr: 5550K .......... .......... .......... .......... .......... 3% 109M 15s 2024-03-29T04:08:40.623 INFO:teuthology.orchestra.run.smithi097.stderr: 5600K .......... .......... .......... .......... .......... 3% 63.1M 15s 2024-03-29T04:08:40.623 INFO:teuthology.orchestra.run.smithi097.stderr: 5650K .......... .......... .......... .......... .......... 3% 31.0M 15s 2024-03-29T04:08:40.623 INFO:teuthology.orchestra.run.smithi097.stderr: 5700K .......... .......... .......... .......... .......... 3% 19.8M 15s 2024-03-29T04:08:40.623 INFO:teuthology.orchestra.run.smithi097.stderr: 5750K .......... .......... .......... .......... .......... 3% 31.5M 15s 2024-03-29T04:08:40.623 INFO:teuthology.orchestra.run.smithi097.stderr: 5800K .......... .......... .......... .......... .......... 3% 35.5M 15s 2024-03-29T04:08:40.623 INFO:teuthology.orchestra.run.smithi097.stderr: 5850K .......... .......... .......... .......... .......... 3% 30.3M 15s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 5900K .......... .......... .......... .......... .......... 3% 30.9M 15s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 5950K .......... .......... .......... .......... .......... 3% 37.8M 15s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 6000K .......... .......... .......... .......... .......... 3% 16.9M 15s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 6050K .......... .......... .......... .......... .......... 3% 37.5M 15s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 6100K .......... .......... .......... .......... .......... 3% 29.9M 15s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 6150K .......... .......... .......... .......... .......... 3% 32.0M 14s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 6200K .......... .......... .......... .......... .......... 3% 37.4M 14s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 6250K .......... .......... .......... .......... .......... 3% 30.0M 14s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 6300K .......... .......... .......... .......... .......... 3% 20.2M 14s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 6350K .......... .......... .......... .......... .......... 3% 31.5M 14s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 6400K .......... .......... .......... .......... .......... 3% 26.4M 14s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 6450K .......... .......... .......... .......... .......... 3% 28.6M 14s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 6500K .......... .......... .......... .......... .......... 3% 84.0M 14s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 6550K .......... .......... .......... .......... .......... 3% 26.6M 14s 2024-03-29T04:08:40.624 INFO:teuthology.orchestra.run.smithi097.stderr: 6600K .......... .......... .......... .......... .......... 3% 18.5M 14s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 6650K .......... .......... .......... .......... .......... 3% 84.7M 14s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 6700K .......... .......... .......... .......... .......... 4% 24.3M 14s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 6750K .......... .......... .......... .......... .......... 4% 25.4M 14s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 6800K .......... .......... .......... .......... .......... 4% 19.6M 14s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 6850K .......... .......... .......... .......... .......... 4% 34.8M 13s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 6900K .......... .......... .......... .......... .......... 4% 26.9M 13s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 6950K .......... .......... .......... .......... .......... 4% 58.2M 13s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 7000K .......... .......... .......... .......... .......... 4% 27.1M 13s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 7050K .......... .......... .......... .......... .......... 4% 24.5M 13s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 7100K .......... .......... .......... .......... .......... 4% 52.4M 13s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 7150K .......... .......... .......... .......... .......... 4% 17.3M 13s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 7200K .......... .......... .......... .......... .......... 4% 60.1M 13s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 7250K .......... .......... .......... .......... .......... 4% 23.4M 13s 2024-03-29T04:08:40.625 INFO:teuthology.orchestra.run.smithi097.stderr: 7300K .......... .......... .......... .......... .......... 4% 27.5M 13s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 7350K .......... .......... .......... .......... .......... 4% 32.5M 13s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 7400K .......... .......... .......... .......... .......... 4% 32.1M 13s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 7450K .......... .......... .......... .......... .......... 4% 33.2M 13s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 7500K .......... .......... .......... .......... .......... 4% 29.1M 13s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 7550K .......... .......... .......... .......... .......... 4% 22.7M 13s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 7600K .......... .......... .......... .......... .......... 4% 18.8M 13s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 7650K .......... .......... .......... .......... .......... 4% 40.9M 13s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 7700K .......... .......... .......... .......... .......... 4% 56.8M 12s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 7750K .......... .......... .......... .......... .......... 4% 23.9M 12s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 7800K .......... .......... .......... .......... .......... 4% 32.8M 12s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 7850K .......... .......... .......... .......... .......... 4% 30.7M 12s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 7900K .......... .......... .......... .......... .......... 4% 27.4M 12s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 7950K .......... .......... .......... .......... .......... 4% 33.2M 12s 2024-03-29T04:08:40.626 INFO:teuthology.orchestra.run.smithi097.stderr: 8000K .......... .......... .......... .......... .......... 4% 26.2M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8050K .......... .......... .......... .......... .......... 4% 25.6M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8100K .......... .......... .......... .......... .......... 4% 43.7M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8150K .......... .......... .......... .......... .......... 4% 19.0M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8200K .......... .......... .......... .......... .......... 4% 62.1M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8250K .......... .......... .......... .......... .......... 4% 27.2M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8300K .......... .......... .......... .......... .......... 4% 25.3M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8350K .......... .......... .......... .......... .......... 4% 25.6M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8400K .......... .......... .......... .......... .......... 5% 24.1M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8450K .......... .......... .......... .......... .......... 5% 59.1M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8500K .......... .......... .......... .......... .......... 5% 25.0M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8550K .......... .......... .......... .......... .......... 5% 22.7M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8600K .......... .......... .......... .......... .......... 5% 33.4M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8650K .......... .......... .......... .......... .......... 5% 37.5M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8700K .......... .......... .......... .......... .......... 5% 32.1M 12s 2024-03-29T04:08:40.627 INFO:teuthology.orchestra.run.smithi097.stderr: 8750K .......... .......... .......... .......... .......... 5% 23.5M 12s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 8800K .......... .......... .......... .......... .......... 5% 25.6M 12s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 8850K .......... .......... .......... .......... .......... 5% 39.7M 12s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 8900K .......... .......... .......... .......... .......... 5% 30.9M 11s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 8950K .......... .......... .......... .......... .......... 5% 30.9M 11s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 9000K .......... .......... .......... .......... .......... 5% 23.0M 11s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 9050K .......... .......... .......... .......... .......... 5% 25.7M 11s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 9100K .......... .......... .......... .......... .......... 5% 45.9M 11s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 9150K .......... .......... .......... .......... .......... 5% 20.2M 11s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 9200K .......... .......... .......... .......... .......... 5% 46.9M 11s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 9250K .......... .......... .......... .......... .......... 5% 23.7M 11s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 9300K .......... .......... .......... .......... .......... 5% 25.6M 11s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 9350K .......... .......... .......... .......... .......... 5% 16.5M 11s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 9400K .......... .......... .......... .......... .......... 5% 15.3M 11s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 9450K .......... .......... .......... .......... .......... 5% 23.2M 11s 2024-03-29T04:08:40.628 INFO:teuthology.orchestra.run.smithi097.stderr: 9500K .......... .......... .......... .......... .......... 5% 11.8M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 9550K .......... .......... .......... .......... .......... 5% 22.1M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 9600K .......... .......... .......... .......... .......... 5% 18.1M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 9650K .......... .......... .......... .......... .......... 5% 17.9M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 9700K .......... .......... .......... .......... .......... 5% 33.0M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 9750K .......... .......... .......... .......... .......... 5% 20.3M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 9800K .......... .......... .......... .......... .......... 5% 98.0M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 9850K .......... .......... .......... .......... .......... 5% 82.8M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 9900K .......... .......... .......... .......... .......... 5% 105M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 9950K .......... .......... .......... .......... .......... 5% 108M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 10000K .......... .......... .......... .......... .......... 5% 94.7M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 10050K .......... .......... .......... .......... .......... 5% 93.9M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 10100K .......... .......... .......... .......... .......... 6% 88.3M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 10150K .......... .......... .......... .......... .......... 6% 102M 11s 2024-03-29T04:08:40.629 INFO:teuthology.orchestra.run.smithi097.stderr: 10200K .......... .......... .......... .......... .......... 6% 33.8M 11s 2024-03-29T04:08:40.630 INFO:teuthology.orchestra.run.smithi097.stderr: 10250K .......... .......... .......... .......... .......... 6% 26.8M 11s 2024-03-29T04:08:40.630 INFO:teuthology.orchestra.run.smithi097.stderr: 10300K .......... .......... .......... .......... .......... 6% 20.2M 11s 2024-03-29T04:08:40.630 INFO:teuthology.orchestra.run.smithi097.stderr: 10350K .......... .......... .......... .......... .......... 6% 61.3M 11s 2024-03-29T04:08:40.630 INFO:teuthology.orchestra.run.smithi097.stderr: 10400K .......... .......... .......... .......... .......... 6% 22.4M 10s 2024-03-29T04:08:40.630 INFO:teuthology.orchestra.run.smithi097.stderr: 10450K .......... .......... .......... .......... .......... 6% 36.9M 10s 2024-03-29T04:08:40.630 INFO:teuthology.orchestra.run.smithi097.stderr: 10500K .......... .......... .......... .......... .......... 6% 30.0M 10s 2024-03-29T04:08:40.630 INFO:teuthology.orchestra.run.smithi097.stderr: 10550K .......... .......... .......... .......... .......... 6% 28.8M 10s 2024-03-29T04:08:40.630 INFO:teuthology.orchestra.run.smithi097.stderr: 10600K .......... .......... .......... .......... .......... 6% 25.6M 10s 2024-03-29T04:08:40.630 INFO:teuthology.orchestra.run.smithi097.stderr: 10650K .......... .......... .......... .......... .......... 6% 23.4M 10s 2024-03-29T04:08:40.630 INFO:teuthology.orchestra.run.smithi097.stderr: 10700K .......... .......... .......... .......... .......... 6% 39.6M 10s 2024-03-29T04:08:40.630 INFO:teuthology.orchestra.run.smithi097.stderr: 10750K .......... .......... .......... .......... .......... 6% 27.7M 10s 2024-03-29T04:08:40.630 INFO:teuthology.orchestra.run.smithi097.stderr: 10800K .......... .......... .......... .......... .......... 6% 32.9M 10s 2024-03-29T04:08:40.630 INFO:teuthology.orchestra.run.smithi097.stderr: 10850K .......... .......... .......... .......... .......... 6% 24.8M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 10900K .......... .......... .......... .......... .......... 6% 34.6M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 10950K .......... .......... .......... .......... .......... 6% 34.2M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 11000K .......... .......... .......... .......... .......... 6% 27.3M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 11050K .......... .......... .......... .......... .......... 6% 22.6M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 11100K .......... .......... .......... .......... .......... 6% 32.5M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 11150K .......... .......... .......... .......... .......... 6% 23.6M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 11200K .......... .......... .......... .......... .......... 6% 42.7M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 11250K .......... .......... .......... .......... .......... 6% 24.5M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 11300K .......... .......... .......... .......... .......... 6% 37.3M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 11350K .......... .......... .......... .......... .......... 6% 22.9M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 11400K .......... .......... .......... .......... .......... 6% 25.0M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 11450K .......... .......... .......... .......... .......... 6% 27.2M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 11500K .......... .......... .......... .......... .......... 6% 41.7M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 11550K .......... .......... .......... .......... .......... 6% 24.5M 10s 2024-03-29T04:08:40.631 INFO:teuthology.orchestra.run.smithi097.stderr: 11600K .......... .......... .......... .......... .......... 6% 38.4M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 11650K .......... .......... .......... .......... .......... 6% 24.1M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 11700K .......... .......... .......... .......... .......... 6% 24.9M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 11750K .......... .......... .......... .......... .......... 7% 44.2M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 11800K .......... .......... .......... .......... .......... 7% 24.7M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 11850K .......... .......... .......... .......... .......... 7% 33.9M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 11900K .......... .......... .......... .......... .......... 7% 24.6M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 11950K .......... .......... .......... .......... .......... 7% 25.7M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 12000K .......... .......... .......... .......... .......... 7% 43.1M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 12050K .......... .......... .......... .......... .......... 7% 26.5M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 12100K .......... .......... .......... .......... .......... 7% 30.8M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 12150K .......... .......... .......... .......... .......... 7% 39.2M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 12200K .......... .......... .......... .......... .......... 7% 14.9M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 12250K .......... .......... .......... .......... .......... 7% 106M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 12300K .......... .......... .......... .......... .......... 7% 27.8M 10s 2024-03-29T04:08:40.632 INFO:teuthology.orchestra.run.smithi097.stderr: 12350K .......... .......... .......... .......... .......... 7% 30.2M 10s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 12400K .......... .......... .......... .......... .......... 7% 24.5M 10s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 12450K .......... .......... .......... .......... .......... 7% 23.5M 10s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 12500K .......... .......... .......... .......... .......... 7% 52.5M 9s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 12550K .......... .......... .......... .......... .......... 7% 24.3M 9s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 12600K .......... .......... .......... .......... .......... 7% 34.1M 9s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 12650K .......... .......... .......... .......... .......... 7% 30.0M 9s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 12700K .......... .......... .......... .......... .......... 7% 33.3M 9s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 12750K .......... .......... .......... .......... .......... 7% 26.9M 9s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 12800K .......... .......... .......... .......... .......... 7% 26.6M 9s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 12850K .......... .......... .......... .......... .......... 7% 24.9M 9s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 12900K .......... .......... .......... .......... .......... 7% 46.6M 9s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 12950K .......... .......... .......... .......... .......... 7% 23.5M 9s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 13000K .......... .......... .......... .......... .......... 7% 35.8M 9s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 13050K .......... .......... .......... .......... .......... 7% 21.9M 9s 2024-03-29T04:08:40.633 INFO:teuthology.orchestra.run.smithi097.stderr: 13100K .......... .......... .......... .......... .......... 7% 36.3M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13150K .......... .......... .......... .......... .......... 7% 23.3M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13200K .......... .......... .......... .......... .......... 7% 27.7M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13250K .......... .......... .......... .......... .......... 7% 43.1M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13300K .......... .......... .......... .......... .......... 7% 27.4M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13350K .......... .......... .......... .......... .......... 7% 23.6M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13400K .......... .......... .......... .......... .......... 7% 34.2M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13450K .......... .......... .......... .......... .......... 8% 22.5M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13500K .......... .......... .......... .......... .......... 8% 61.1M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13550K .......... .......... .......... .......... .......... 8% 28.2M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13600K .......... .......... .......... .......... .......... 8% 27.6M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13650K .......... .......... .......... .......... .......... 8% 24.6M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13700K .......... .......... .......... .......... .......... 8% 18.6M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13750K .......... .......... .......... .......... .......... 8% 106M 9s 2024-03-29T04:08:40.634 INFO:teuthology.orchestra.run.smithi097.stderr: 13800K .......... .......... .......... .......... .......... 8% 26.7M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 13850K .......... .......... .......... .......... .......... 8% 31.9M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 13900K .......... .......... .......... .......... .......... 8% 24.2M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 13950K .......... .......... .......... .......... .......... 8% 33.2M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 14000K .......... .......... .......... .......... .......... 8% 33.5M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 14050K .......... .......... .......... .......... .......... 8% 25.3M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 14100K .......... .......... .......... .......... .......... 8% 25.9M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 14150K .......... .......... .......... .......... .......... 8% 25.0M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 14200K .......... .......... .......... .......... .......... 8% 48.1M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 14250K .......... .......... .......... .......... .......... 8% 32.4M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 14300K .......... .......... .......... .......... .......... 8% 23.4M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 14350K .......... .......... .......... .......... .......... 8% 24.5M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 14400K .......... .......... .......... .......... .......... 8% 45.5M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 14450K .......... .......... .......... .......... .......... 8% 23.5M 9s 2024-03-29T04:08:40.635 INFO:teuthology.orchestra.run.smithi097.stderr: 14500K .......... .......... .......... .......... .......... 8% 45.3M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 14550K .......... .......... .......... .......... .......... 8% 23.6M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 14600K .......... .......... .......... .......... .......... 8% 31.7M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 14650K .......... .......... .......... .......... .......... 8% 23.3M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 14700K .......... .......... .......... .......... .......... 8% 23.0M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 14750K .......... .......... .......... .......... .......... 8% 66.1M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 14800K .......... .......... .......... .......... .......... 8% 24.4M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 14850K .......... .......... .......... .......... .......... 8% 30.7M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 14900K .......... .......... .......... .......... .......... 8% 23.7M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 14950K .......... .......... .......... .......... .......... 8% 25.1M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 15000K .......... .......... .......... .......... .......... 8% 85.5M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 15050K .......... .......... .......... .......... .......... 8% 24.4M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 15100K .......... .......... .......... .......... .......... 8% 22.9M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 15150K .......... .......... .......... .......... .......... 9% 33.9M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 15200K .......... .......... .......... .......... .......... 9% 18.8M 9s 2024-03-29T04:08:40.636 INFO:teuthology.orchestra.run.smithi097.stderr: 15250K .......... .......... .......... .......... .......... 9% 30.2M 9s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15300K .......... .......... .......... .......... .......... 9% 28.3M 9s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15350K .......... .......... .......... .......... .......... 9% 36.0M 9s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15400K .......... .......... .......... .......... .......... 9% 32.5M 9s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15450K .......... .......... .......... .......... .......... 9% 50.2M 9s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15500K .......... .......... .......... .......... .......... 9% 23.7M 9s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15550K .......... .......... .......... .......... .......... 9% 31.0M 8s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15600K .......... .......... .......... .......... .......... 9% 25.4M 8s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15650K .......... .......... .......... .......... .......... 9% 46.2M 8s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15700K .......... .......... .......... .......... .......... 9% 22.1M 8s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15750K .......... .......... .......... .......... .......... 9% 37.5M 8s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15800K .......... .......... .......... .......... .......... 9% 14.5M 8s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15850K .......... .......... .......... .......... .......... 9% 21.8M 8s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15900K .......... .......... .......... .......... .......... 9% 23.6M 8s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 15950K .......... .......... .......... .......... .......... 9% 13.3M 8s 2024-03-29T04:08:40.637 INFO:teuthology.orchestra.run.smithi097.stderr: 16000K .......... .......... .......... .......... .......... 9% 28.0M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16050K .......... .......... .......... .......... .......... 9% 18.0M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16100K .......... .......... .......... .......... .......... 9% 23.0M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16150K .......... .......... .......... .......... .......... 9% 17.7M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16200K .......... .......... .......... .......... .......... 9% 12.8M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16250K .......... .......... .......... .......... .......... 9% 58.3M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16300K .......... .......... .......... .......... .......... 9% 42.0M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16350K .......... .......... .......... .......... .......... 9% 62.4M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16400K .......... .......... .......... .......... .......... 9% 87.7M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16450K .......... .......... .......... .......... .......... 9% 88.7M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16500K .......... .......... .......... .......... .......... 9% 106M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16550K .......... .......... .......... .......... .......... 9% 54.2M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16600K .......... .......... .......... .......... .......... 9% 85.2M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16650K .......... .......... .......... .......... .......... 9% 90.8M 8s 2024-03-29T04:08:40.638 INFO:teuthology.orchestra.run.smithi097.stderr: 16700K .......... .......... .......... .......... .......... 9% 55.1M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 16750K .......... .......... .......... .......... .......... 9% 20.5M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 16800K .......... .......... .......... .......... .......... 9% 36.3M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 16850K .......... .......... .......... .......... .......... 10% 23.9M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 16900K .......... .......... .......... .......... .......... 10% 24.1M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 16950K .......... .......... .......... .......... .......... 10% 44.9M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 17000K .......... .......... .......... .......... .......... 10% 23.9M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 17050K .......... .......... .......... .......... .......... 10% 35.6M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 17100K .......... .......... .......... .......... .......... 10% 24.6M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 17150K .......... .......... .......... .......... .......... 10% 41.5M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 17200K .......... .......... .......... .......... .......... 10% 21.0M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 17250K .......... .......... .......... .......... .......... 10% 26.7M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 17300K .......... .......... .......... .......... .......... 10% 47.6M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 17350K .......... .......... .......... .......... .......... 10% 24.7M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 17400K .......... .......... .......... .......... .......... 10% 35.1M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 17450K .......... .......... .......... .......... .......... 10% 24.5M 8s 2024-03-29T04:08:40.639 INFO:teuthology.orchestra.run.smithi097.stderr: 17500K .......... .......... .......... .......... .......... 10% 49.7M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 17550K .......... .......... .......... .......... .......... 10% 25.1M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 17600K .......... .......... .......... .......... .......... 10% 28.0M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 17650K .......... .......... .......... .......... .......... 10% 22.5M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 17700K .......... .......... .......... .......... .......... 10% 39.0M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 17750K .......... .......... .......... .......... .......... 10% 22.4M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 17800K .......... .......... .......... .......... .......... 10% 38.9M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 17850K .......... .......... .......... .......... .......... 10% 25.7M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 17900K .......... .......... .......... .......... .......... 10% 31.5M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 17950K .......... .......... .......... .......... .......... 10% 24.4M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 18000K .......... .......... .......... .......... .......... 10% 51.5M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 18050K .......... .......... .......... .......... .......... 10% 27.9M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 18100K .......... .......... .......... .......... .......... 10% 22.4M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 18150K .......... .......... .......... .......... .......... 10% 28.3M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 18200K .......... .......... .......... .......... .......... 10% 32.9M 8s 2024-03-29T04:08:40.640 INFO:teuthology.orchestra.run.smithi097.stderr: 18250K .......... .......... .......... .......... .......... 10% 23.9M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18300K .......... .......... .......... .......... .......... 10% 45.3M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18350K .......... .......... .......... .......... .......... 10% 24.3M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18400K .......... .......... .......... .......... .......... 10% 38.7M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18450K .......... .......... .......... .......... .......... 10% 24.3M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18500K .......... .......... .......... .......... .......... 11% 24.3M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18550K .......... .......... .......... .......... .......... 11% 43.0M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18600K .......... .......... .......... .......... .......... 11% 25.2M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18650K .......... .......... .......... .......... .......... 11% 22.2M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18700K .......... .......... .......... .......... .......... 11% 40.7M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18750K .......... .......... .......... .......... .......... 11% 25.2M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18800K .......... .......... .......... .......... .......... 11% 47.3M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18850K .......... .......... .......... .......... .......... 11% 27.8M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18900K .......... .......... .......... .......... .......... 11% 24.1M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 18950K .......... .......... .......... .......... .......... 11% 24.5M 8s 2024-03-29T04:08:40.641 INFO:teuthology.orchestra.run.smithi097.stderr: 19000K .......... .......... .......... .......... .......... 11% 43.9M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19050K .......... .......... .......... .......... .......... 11% 33.9M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19100K .......... .......... .......... .......... .......... 11% 23.0M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19150K .......... .......... .......... .......... .......... 11% 14.8M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19200K .......... .......... .......... .......... .......... 11% 385M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19250K .......... .......... .......... .......... .......... 11% 40.2M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19300K .......... .......... .......... .......... .......... 11% 26.1M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19350K .......... .......... .......... .......... .......... 11% 32.6M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19400K .......... .......... .......... .......... .......... 11% 23.3M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19450K .......... .......... .......... .......... .......... 11% 24.2M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19500K .......... .......... .......... .......... .......... 11% 43.1M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19550K .......... .......... .......... .......... .......... 11% 33.6M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19600K .......... .......... .......... .......... .......... 11% 23.4M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19650K .......... .......... .......... .......... .......... 11% 35.7M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19700K .......... .......... .......... .......... .......... 11% 24.7M 8s 2024-03-29T04:08:40.642 INFO:teuthology.orchestra.run.smithi097.stderr: 19750K .......... .......... .......... .......... .......... 11% 23.0M 8s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 19800K .......... .......... .......... .......... .......... 11% 60.7M 8s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 19850K .......... .......... .......... .......... .......... 11% 23.6M 8s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 19900K .......... .......... .......... .......... .......... 11% 24.0M 8s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 19950K .......... .......... .......... .......... .......... 11% 35.8M 8s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 20000K .......... .......... .......... .......... .......... 11% 22.6M 8s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 20050K .......... .......... .......... .......... .......... 11% 58.6M 8s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 20100K .......... .......... .......... .......... .......... 11% 24.8M 8s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 20150K .......... .......... .......... .......... .......... 11% 30.3M 8s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 20200K .......... .......... .......... .......... .......... 12% 20.6M 7s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 20250K .......... .......... .......... .......... .......... 12% 63.7M 7s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 20300K .......... .......... .......... .......... .......... 12% 22.1M 7s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 20350K .......... .......... .......... .......... .......... 12% 27.2M 7s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 20400K .......... .......... .......... .......... .......... 12% 38.4M 7s 2024-03-29T04:08:40.643 INFO:teuthology.orchestra.run.smithi097.stderr: 20450K .......... .......... .......... .......... .......... 12% 26.2M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 20500K .......... .......... .......... .......... .......... 12% 26.2M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 20550K .......... .......... .......... .......... .......... 12% 45.4M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 20600K .......... .......... .......... .......... .......... 12% 24.4M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 20650K .......... .......... .......... .......... .......... 12% 24.2M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 20700K .......... .......... .......... .......... .......... 12% 26.4M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 20750K .......... .......... .......... .......... .......... 12% 54.0M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 20800K .......... .......... .......... .......... .......... 12% 32.0M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 20850K .......... .......... .......... .......... .......... 12% 23.7M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 20900K .......... .......... .......... .......... .......... 12% 23.8M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 20950K .......... .......... .......... .......... .......... 12% 35.3M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 21000K .......... .......... .......... .......... .......... 12% 23.8M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 21050K .......... .......... .......... .......... .......... 12% 49.8M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 21100K .......... .......... .......... .......... .......... 12% 25.4M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 21150K .......... .......... .......... .......... .......... 12% 32.0M 7s 2024-03-29T04:08:40.644 INFO:teuthology.orchestra.run.smithi097.stderr: 21200K .......... .......... .......... .......... .......... 12% 24.3M 7s 2024-03-29T04:08:40.645 INFO:teuthology.orchestra.run.smithi097.stderr: 21250K .......... .......... .......... .......... .......... 12% 18.4M 7s 2024-03-29T04:08:40.645 INFO:teuthology.orchestra.run.smithi097.stderr: 21300K .......... .......... .......... .......... .......... 12% 122M 7s 2024-03-29T04:08:40.645 INFO:teuthology.orchestra.run.smithi097.stderr: 21350K .......... .......... .......... .......... .......... 12% 25.5M 7s 2024-03-29T04:08:40.645 INFO:teuthology.orchestra.run.smithi097.stderr: 21400K .......... .......... .......... .......... .......... 12% 36.0M 7s 2024-03-29T04:08:40.645 INFO:teuthology.orchestra.run.smithi097.stderr: 21450K .......... .......... .......... .......... .......... 12% 21.0M 7s 2024-03-29T04:08:40.645 INFO:teuthology.orchestra.run.smithi097.stderr: 21500K .......... .......... .......... .......... .......... 12% 28.7M 7s 2024-03-29T04:08:40.645 INFO:teuthology.orchestra.run.smithi097.stderr: 21550K .......... .......... .......... .......... .......... 12% 47.6M 7s 2024-03-29T04:08:40.646 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:08:41.155 INFO:teuthology.orchestra.run.smithi097.stderr: 21600K .......... .......... .......... .......... .......... 12% 23.6M 7s 2024-03-29T04:08:41.155 INFO:teuthology.orchestra.run.smithi097.stderr: 21650K .......... .......... .......... .......... .......... 12% 25.1M 7s 2024-03-29T04:08:41.155 INFO:teuthology.orchestra.run.smithi097.stderr: 21700K .......... .......... .......... .......... .......... 12% 25.1M 7s 2024-03-29T04:08:41.155 INFO:teuthology.orchestra.run.smithi097.stderr: 21750K .......... .......... .......... .......... .......... 12% 39.4M 7s 2024-03-29T04:08:41.156 INFO:teuthology.orchestra.run.smithi097.stderr: 21800K .......... .......... .......... .......... .......... 12% 43.0M 7s 2024-03-29T04:08:41.156 INFO:teuthology.orchestra.run.smithi097.stderr: 21850K .......... .......... .......... .......... .......... 12% 22.8M 7s 2024-03-29T04:08:41.156 INFO:teuthology.orchestra.run.smithi097.stderr: 21900K .......... .......... .......... .......... .......... 13% 28.2M 7s 2024-03-29T04:08:41.156 INFO:teuthology.orchestra.run.smithi097.stderr: 21950K .......... .......... .......... .......... .......... 13% 22.3M 7s 2024-03-29T04:08:41.156 INFO:teuthology.orchestra.run.smithi097.stderr: 22000K .......... .......... .......... .......... .......... 13% 56.9M 7s 2024-03-29T04:08:41.156 INFO:teuthology.orchestra.run.smithi097.stderr: 22050K .......... .......... .......... .......... .......... 13% 16.6M 7s 2024-03-29T04:08:41.156 INFO:teuthology.orchestra.run.smithi097.stderr: 22100K .......... .......... .......... .......... .......... 13% 38.7M 7s 2024-03-29T04:08:41.157 INFO:teuthology.orchestra.run.smithi097.stderr: 22150K .......... .......... .......... .......... .......... 13% 26.3M 7s 2024-03-29T04:08:41.157 INFO:teuthology.orchestra.run.smithi097.stderr: 22200K .......... .......... .......... .......... .......... 13% 15.4M 7s 2024-03-29T04:08:41.157 INFO:teuthology.orchestra.run.smithi097.stderr: 22250K .......... .......... .......... .......... .......... 13% 47.2M 7s 2024-03-29T04:08:41.157 INFO:teuthology.orchestra.run.smithi097.stderr: 22300K .......... .......... .......... .......... .......... 13% 28.9M 7s 2024-03-29T04:08:41.157 INFO:teuthology.orchestra.run.smithi097.stderr: 22350K .......... .......... .......... .......... .......... 13% 45.5M 7s 2024-03-29T04:08:41.157 INFO:teuthology.orchestra.run.smithi097.stderr: 22400K .......... .......... .......... .......... .......... 13% 30.6M 7s 2024-03-29T04:08:41.157 INFO:teuthology.orchestra.run.smithi097.stderr: 22450K .......... .......... .......... .......... .......... 13% 32.0M 7s 2024-03-29T04:08:41.158 INFO:teuthology.orchestra.run.smithi097.stderr: 22500K .......... .......... .......... .......... .......... 13% 23.0M 7s 2024-03-29T04:08:41.158 INFO:teuthology.orchestra.run.smithi097.stderr: 22550K .......... .......... .......... .......... .......... 13% 26.7M 7s 2024-03-29T04:08:41.158 INFO:teuthology.orchestra.run.smithi097.stderr: 22600K .......... .......... .......... .......... .......... 13% 48.7M 7s 2024-03-29T04:08:41.158 INFO:teuthology.orchestra.run.smithi097.stderr: 22650K .......... .......... .......... .......... .......... 13% 30.5M 7s 2024-03-29T04:08:41.158 INFO:teuthology.orchestra.run.smithi097.stderr: 22700K .......... .......... .......... .......... .......... 13% 22.2M 7s 2024-03-29T04:08:41.159 INFO:teuthology.orchestra.run.smithi097.stderr: 22750K .......... .......... .......... .......... .......... 13% 31.0M 7s 2024-03-29T04:08:41.159 INFO:teuthology.orchestra.run.smithi097.stderr: 22800K .......... .......... .......... .......... .......... 13% 45.6M 7s 2024-03-29T04:08:41.159 INFO:teuthology.orchestra.run.smithi097.stderr: 22850K .......... .......... .......... .......... .......... 13% 30.0M 7s 2024-03-29T04:08:41.159 INFO:teuthology.orchestra.run.smithi097.stderr: 22900K .......... .......... .......... .......... .......... 13% 21.8M 7s 2024-03-29T04:08:41.159 INFO:teuthology.orchestra.run.smithi097.stderr: 22950K .......... .......... .......... .......... .......... 13% 27.0M 7s 2024-03-29T04:08:41.159 INFO:teuthology.orchestra.run.smithi097.stderr: 23000K .......... .......... .......... .......... .......... 13% 26.3M 7s 2024-03-29T04:08:41.160 INFO:teuthology.orchestra.run.smithi097.stderr: 23050K .......... .......... .......... .......... .......... 13% 18.6M 7s 2024-03-29T04:08:41.160 INFO:teuthology.orchestra.run.smithi097.stderr: 23100K .......... .......... .......... .......... .......... 13% 45.8M 7s 2024-03-29T04:08:41.160 INFO:teuthology.orchestra.run.smithi097.stderr: 23150K .......... .......... .......... .......... .......... 13% 17.2M 7s 2024-03-29T04:08:41.160 INFO:teuthology.orchestra.run.smithi097.stderr: 23200K .......... .......... .......... .......... .......... 13% 33.3M 7s 2024-03-29T04:08:41.160 INFO:teuthology.orchestra.run.smithi097.stderr: 23250K .......... .......... .......... .......... .......... 13% 66.3M 7s 2024-03-29T04:08:41.160 INFO:teuthology.orchestra.run.smithi097.stderr: 23300K .......... .......... .......... .......... .......... 13% 77.9M 7s 2024-03-29T04:08:41.161 INFO:teuthology.orchestra.run.smithi097.stderr: 23350K .......... .......... .......... .......... .......... 13% 63.2M 7s 2024-03-29T04:08:41.161 INFO:teuthology.orchestra.run.smithi097.stderr: 23400K .......... .......... .......... .......... .......... 13% 24.9M 7s 2024-03-29T04:08:41.161 INFO:teuthology.orchestra.run.smithi097.stderr: 23450K .......... .......... .......... .......... .......... 13% 24.6M 7s 2024-03-29T04:08:41.161 INFO:teuthology.orchestra.run.smithi097.stderr: 23500K .......... .......... .......... .......... .......... 13% 31.4M 7s 2024-03-29T04:08:41.161 INFO:teuthology.orchestra.run.smithi097.stderr: 23550K .......... .......... .......... .......... .......... 14% 27.4M 7s 2024-03-29T04:08:41.161 INFO:teuthology.orchestra.run.smithi097.stderr: 23600K .......... .......... .......... .......... .......... 14% 37.9M 7s 2024-03-29T04:08:41.162 INFO:teuthology.orchestra.run.smithi097.stderr: 23650K .......... .......... .......... .......... .......... 14% 24.7M 7s 2024-03-29T04:08:41.162 INFO:teuthology.orchestra.run.smithi097.stderr: 23700K .......... .......... .......... .......... .......... 14% 35.5M 7s 2024-03-29T04:08:41.162 INFO:teuthology.orchestra.run.smithi097.stderr: 23750K .......... .......... .......... .......... .......... 14% 24.4M 7s 2024-03-29T04:08:41.162 INFO:teuthology.orchestra.run.smithi097.stderr: 23800K .......... .......... .......... .......... .......... 14% 23.9M 7s 2024-03-29T04:08:41.162 INFO:teuthology.orchestra.run.smithi097.stderr: 23850K .......... .......... .......... .......... .......... 14% 45.8M 7s 2024-03-29T04:08:41.162 INFO:teuthology.orchestra.run.smithi097.stderr: 23900K .......... .......... .......... .......... .......... 14% 26.9M 7s 2024-03-29T04:08:41.163 INFO:teuthology.orchestra.run.smithi097.stderr: 23950K .......... .......... .......... .......... .......... 14% 30.4M 7s 2024-03-29T04:08:41.163 INFO:teuthology.orchestra.run.smithi097.stderr: 24000K .......... .......... .......... .......... .......... 14% 21.5M 7s 2024-03-29T04:08:41.163 INFO:teuthology.orchestra.run.smithi097.stderr: 24050K .......... .......... .......... .......... .......... 14% 28.4M 7s 2024-03-29T04:08:41.163 INFO:teuthology.orchestra.run.smithi097.stderr: 24100K .......... .......... .......... .......... .......... 14% 67.5M 7s 2024-03-29T04:08:41.163 INFO:teuthology.orchestra.run.smithi097.stderr: 24150K .......... .......... .......... .......... .......... 14% 21.9M 7s 2024-03-29T04:08:41.163 INFO:teuthology.orchestra.run.smithi097.stderr: 24200K .......... .......... .......... .......... .......... 14% 33.2M 7s 2024-03-29T04:08:41.164 INFO:teuthology.orchestra.run.smithi097.stderr: 24250K .......... .......... .......... .......... .......... 14% 23.0M 7s 2024-03-29T04:08:41.164 INFO:teuthology.orchestra.run.smithi097.stderr: 24300K .......... .......... .......... .......... .......... 14% 36.4M 7s 2024-03-29T04:08:41.164 INFO:teuthology.orchestra.run.smithi097.stderr: 24350K .......... .......... .......... .......... .......... 14% 33.0M 7s 2024-03-29T04:08:41.164 INFO:teuthology.orchestra.run.smithi097.stderr: 24400K .......... .......... .......... .......... .......... 14% 25.0M 7s 2024-03-29T04:08:41.164 INFO:teuthology.orchestra.run.smithi097.stderr: 24450K .......... .......... .......... .......... .......... 14% 24.0M 7s 2024-03-29T04:08:41.164 INFO:teuthology.orchestra.run.smithi097.stderr: 24500K .......... .......... .......... .......... .......... 14% 38.9M 7s 2024-03-29T04:08:41.165 INFO:teuthology.orchestra.run.smithi097.stderr: 24550K .......... .......... .......... .......... .......... 14% 23.8M 7s 2024-03-29T04:08:41.165 INFO:teuthology.orchestra.run.smithi097.stderr: 24600K .......... .......... .......... .......... .......... 14% 46.6M 7s 2024-03-29T04:08:41.165 INFO:teuthology.orchestra.run.smithi097.stderr: 24650K .......... .......... .......... .......... .......... 14% 24.2M 7s 2024-03-29T04:08:41.165 INFO:teuthology.orchestra.run.smithi097.stderr: 24700K .......... .......... .......... .......... .......... 14% 24.8M 7s 2024-03-29T04:08:41.165 INFO:teuthology.orchestra.run.smithi097.stderr: 24750K .......... .......... .......... .......... .......... 14% 58.0M 7s 2024-03-29T04:08:41.165 INFO:teuthology.orchestra.run.smithi097.stderr: 24800K .......... .......... .......... .......... .......... 14% 23.4M 7s 2024-03-29T04:08:41.166 INFO:teuthology.orchestra.run.smithi097.stderr: 24850K .......... .......... .......... .......... .......... 14% 32.3M 7s 2024-03-29T04:08:41.166 INFO:teuthology.orchestra.run.smithi097.stderr: 24900K .......... .......... .......... .......... .......... 14% 25.5M 7s 2024-03-29T04:08:41.166 INFO:teuthology.orchestra.run.smithi097.stderr: 24950K .......... .......... .......... .......... .......... 14% 24.3M 7s 2024-03-29T04:08:41.166 INFO:teuthology.orchestra.run.smithi097.stderr: 25000K .......... .......... .......... .......... .......... 14% 36.4M 7s 2024-03-29T04:08:41.166 INFO:teuthology.orchestra.run.smithi097.stderr: 25050K .......... .......... .......... .......... .......... 14% 22.8M 7s 2024-03-29T04:08:41.166 INFO:teuthology.orchestra.run.smithi097.stderr: 25100K .......... .......... .......... .......... .......... 14% 49.6M 7s 2024-03-29T04:08:41.166 INFO:teuthology.orchestra.run.smithi097.stderr: 25150K .......... .......... .......... .......... .......... 14% 24.3M 7s 2024-03-29T04:08:41.167 INFO:teuthology.orchestra.run.smithi097.stderr: 25200K .......... .......... .......... .......... .......... 14% 35.8M 7s 2024-03-29T04:08:41.167 INFO:teuthology.orchestra.run.smithi097.stderr: 25250K .......... .......... .......... .......... .......... 15% 19.7M 7s 2024-03-29T04:08:41.167 INFO:teuthology.orchestra.run.smithi097.stderr: 25300K .......... .......... .......... .......... .......... 15% 31.9M 7s 2024-03-29T04:08:41.167 INFO:teuthology.orchestra.run.smithi097.stderr: 25350K .......... .......... .......... .......... .......... 15% 48.7M 7s 2024-03-29T04:08:41.167 INFO:teuthology.orchestra.run.smithi097.stderr: 25400K .......... .......... .......... .......... .......... 15% 24.2M 7s 2024-03-29T04:08:41.167 INFO:teuthology.orchestra.run.smithi097.stderr: 25450K .......... .......... .......... .......... .......... 15% 36.1M 7s 2024-03-29T04:08:41.168 INFO:teuthology.orchestra.run.smithi097.stderr: 25500K .......... .......... .......... .......... .......... 15% 31.3M 7s 2024-03-29T04:08:41.168 INFO:teuthology.orchestra.run.smithi097.stderr: 25550K .......... .......... .......... .......... .......... 15% 27.8M 7s 2024-03-29T04:08:41.168 INFO:teuthology.orchestra.run.smithi097.stderr: 25600K .......... .......... .......... .......... .......... 15% 30.5M 7s 2024-03-29T04:08:41.168 INFO:teuthology.orchestra.run.smithi097.stderr: 25650K .......... .......... .......... .......... .......... 15% 23.9M 7s 2024-03-29T04:08:41.168 INFO:teuthology.orchestra.run.smithi097.stderr: 25700K .......... .......... .......... .......... .......... 15% 33.0M 7s 2024-03-29T04:08:41.168 INFO:teuthology.orchestra.run.smithi097.stderr: 25750K .......... .......... .......... .......... .......... 15% 20.7M 7s 2024-03-29T04:08:41.168 INFO:teuthology.orchestra.run.smithi097.stderr: 25800K .......... .......... .......... .......... .......... 15% 33.4M 7s 2024-03-29T04:08:41.169 INFO:teuthology.orchestra.run.smithi097.stderr: 25850K .......... .......... .......... .......... .......... 15% 43.6M 7s 2024-03-29T04:08:41.169 INFO:teuthology.orchestra.run.smithi097.stderr: 25900K .......... .......... .......... .......... .......... 15% 26.7M 7s 2024-03-29T04:08:41.169 INFO:teuthology.orchestra.run.smithi097.stderr: 25950K .......... .......... .......... .......... .......... 15% 29.5M 7s 2024-03-29T04:08:41.169 INFO:teuthology.orchestra.run.smithi097.stderr: 26000K .......... .......... .......... .......... .......... 15% 24.7M 7s 2024-03-29T04:08:41.169 INFO:teuthology.orchestra.run.smithi097.stderr: 26050K .......... .......... .......... .......... .......... 15% 31.6M 7s 2024-03-29T04:08:41.169 INFO:teuthology.orchestra.run.smithi097.stderr: 26100K .......... .......... .......... .......... .......... 15% 34.6M 7s 2024-03-29T04:08:41.170 INFO:teuthology.orchestra.run.smithi097.stderr: 26150K .......... .......... .......... .......... .......... 15% 23.4M 7s 2024-03-29T04:08:41.170 INFO:teuthology.orchestra.run.smithi097.stderr: 26200K .......... .......... .......... .......... .......... 15% 35.5M 7s 2024-03-29T04:08:41.170 INFO:teuthology.orchestra.run.smithi097.stderr: 26250K .......... .......... .......... .......... .......... 15% 27.9M 7s 2024-03-29T04:08:41.170 INFO:teuthology.orchestra.run.smithi097.stderr: 26300K .......... .......... .......... .......... .......... 15% 22.9M 7s 2024-03-29T04:08:41.170 INFO:teuthology.orchestra.run.smithi097.stderr: 26350K .......... .......... .......... .......... .......... 15% 41.4M 7s 2024-03-29T04:08:41.170 INFO:teuthology.orchestra.run.smithi097.stderr: 26400K .......... .......... .......... .......... .......... 15% 25.6M 7s 2024-03-29T04:08:41.171 INFO:teuthology.orchestra.run.smithi097.stderr: 26450K .......... .......... .......... .......... .......... 15% 32.6M 7s 2024-03-29T04:08:41.171 INFO:teuthology.orchestra.run.smithi097.stderr: 26500K .......... .......... .......... .......... .......... 15% 24.6M 7s 2024-03-29T04:08:41.171 INFO:teuthology.orchestra.run.smithi097.stderr: 26550K .......... .......... .......... .......... .......... 15% 23.9M 7s 2024-03-29T04:08:41.171 INFO:teuthology.orchestra.run.smithi097.stderr: 26600K .......... .......... .......... .......... .......... 15% 57.4M 7s 2024-03-29T04:08:41.171 INFO:teuthology.orchestra.run.smithi097.stderr: 26650K .......... .......... .......... .......... .......... 15% 26.1M 7s 2024-03-29T04:08:41.171 INFO:teuthology.orchestra.run.smithi097.stderr: 26700K .......... .......... .......... .......... .......... 15% 21.9M 7s 2024-03-29T04:08:41.171 INFO:teuthology.orchestra.run.smithi097.stderr: 26750K .......... .......... .......... .......... .......... 15% 26.2M 7s 2024-03-29T04:08:41.171 INFO:teuthology.orchestra.run.smithi097.stderr: 26800K .......... .......... .......... .......... .......... 15% 29.2M 7s 2024-03-29T04:08:41.171 INFO:teuthology.orchestra.run.smithi097.stderr: 26850K .......... .......... .......... .......... .......... 15% 48.5M 7s 2024-03-29T04:08:41.171 INFO:teuthology.orchestra.run.smithi097.stderr: 26900K .......... .......... .......... .......... .......... 15% 25.5M 7s 2024-03-29T04:08:41.172 INFO:teuthology.orchestra.run.smithi097.stderr: 26950K .......... .......... .......... .......... .......... 16% 23.3M 7s 2024-03-29T04:08:41.172 INFO:teuthology.orchestra.run.smithi097.stderr: 27000K .......... .......... .......... .......... .......... 16% 21.0M 7s 2024-03-29T04:08:41.172 INFO:teuthology.orchestra.run.smithi097.stderr: 27050K .......... .......... .......... .......... .......... 16% 89.9M 7s 2024-03-29T04:08:41.172 INFO:teuthology.orchestra.run.smithi097.stderr: 27100K .......... .......... .......... .......... .......... 16% 19.8M 7s 2024-03-29T04:08:41.172 INFO:teuthology.orchestra.run.smithi097.stderr: 27150K .......... .......... .......... .......... .......... 16% 50.3M 7s 2024-03-29T04:08:41.172 INFO:teuthology.orchestra.run.smithi097.stderr: 27200K .......... .......... .......... .......... .......... 16% 24.6M 7s 2024-03-29T04:08:41.172 INFO:teuthology.orchestra.run.smithi097.stderr: 27250K .......... .......... .......... .......... .......... 16% 34.8M 7s 2024-03-29T04:08:41.172 INFO:teuthology.orchestra.run.smithi097.stderr: 27300K .......... .......... .......... .......... .......... 16% 26.7M 7s 2024-03-29T04:08:41.172 INFO:teuthology.orchestra.run.smithi097.stderr: 27350K .......... .......... .......... .......... .......... 16% 22.2M 7s 2024-03-29T04:08:41.173 INFO:teuthology.orchestra.run.smithi097.stderr: 27400K .......... .......... .......... .......... .......... 16% 73.9M 7s 2024-03-29T04:08:41.173 INFO:teuthology.orchestra.run.smithi097.stderr: 27450K .......... .......... .......... .......... .......... 16% 20.6M 7s 2024-03-29T04:08:41.173 INFO:teuthology.orchestra.run.smithi097.stderr: 27500K .......... .......... .......... .......... .......... 16% 42.1M 6s 2024-03-29T04:08:41.173 INFO:teuthology.orchestra.run.smithi097.stderr: 27550K .......... .......... .......... .......... .......... 16% 25.5M 6s 2024-03-29T04:08:41.173 INFO:teuthology.orchestra.run.smithi097.stderr: 27600K .......... .......... .......... .......... .......... 16% 23.0M 6s 2024-03-29T04:08:41.173 INFO:teuthology.orchestra.run.smithi097.stderr: 27650K .......... .......... .......... .......... .......... 16% 40.3M 6s 2024-03-29T04:08:41.173 INFO:teuthology.orchestra.run.smithi097.stderr: 27700K .......... .......... .......... .......... .......... 16% 29.0M 6s 2024-03-29T04:08:41.173 INFO:teuthology.orchestra.run.smithi097.stderr: 27750K .......... .......... .......... .......... .......... 16% 28.9M 6s 2024-03-29T04:08:41.174 INFO:teuthology.orchestra.run.smithi097.stderr: 27800K .......... .......... .......... .......... .......... 16% 24.4M 6s 2024-03-29T04:08:41.174 INFO:teuthology.orchestra.run.smithi097.stderr: 27850K .......... .......... .......... .......... .......... 16% 25.2M 6s 2024-03-29T04:08:41.174 INFO:teuthology.orchestra.run.smithi097.stderr: 27900K .......... .......... .......... .......... .......... 16% 48.4M 6s 2024-03-29T04:08:41.174 INFO:teuthology.orchestra.run.smithi097.stderr: 27950K .......... .......... .......... .......... .......... 16% 24.4M 6s 2024-03-29T04:08:41.174 INFO:teuthology.orchestra.run.smithi097.stderr: 28000K .......... .......... .......... .......... .......... 16% 29.4M 6s 2024-03-29T04:08:41.174 INFO:teuthology.orchestra.run.smithi097.stderr: 28050K .......... .......... .......... .......... .......... 16% 27.3M 6s 2024-03-29T04:08:41.175 INFO:teuthology.orchestra.run.smithi097.stderr: 28100K .......... .......... .......... .......... .......... 16% 25.2M 6s 2024-03-29T04:08:41.175 INFO:teuthology.orchestra.run.smithi097.stderr: 28150K .......... .......... .......... .......... .......... 16% 51.3M 6s 2024-03-29T04:08:41.175 INFO:teuthology.orchestra.run.smithi097.stderr: 28200K .......... .......... .......... .......... .......... 16% 23.8M 6s 2024-03-29T04:08:41.175 INFO:teuthology.orchestra.run.smithi097.stderr: 28250K .......... .......... .......... .......... .......... 16% 34.6M 6s 2024-03-29T04:08:41.175 INFO:teuthology.orchestra.run.smithi097.stderr: 28300K .......... .......... .......... .......... .......... 16% 24.8M 6s 2024-03-29T04:08:41.175 INFO:teuthology.orchestra.run.smithi097.stderr: 28350K .......... .......... .......... .......... .......... 16% 35.0M 6s 2024-03-29T04:08:41.175 INFO:teuthology.orchestra.run.smithi097.stderr: 28400K .......... .......... .......... .......... .......... 16% 32.6M 6s 2024-03-29T04:08:41.175 INFO:teuthology.orchestra.run.smithi097.stderr: 28450K .......... .......... .......... .......... .......... 16% 23.0M 6s 2024-03-29T04:08:41.175 INFO:teuthology.orchestra.run.smithi097.stderr: 28500K .......... .......... .......... .......... .......... 16% 28.8M 6s 2024-03-29T04:08:41.175 INFO:teuthology.orchestra.run.smithi097.stderr: 28550K .......... .......... .......... .......... .......... 16% 27.3M 6s 2024-03-29T04:08:41.175 INFO:teuthology.orchestra.run.smithi097.stderr: 28600K .......... .......... .......... .......... .......... 16% 25.9M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 28650K .......... .......... .......... .......... .......... 17% 48.6M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 28700K .......... .......... .......... .......... .......... 17% 25.2M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 28750K .......... .......... .......... .......... .......... 17% 25.6M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 28800K .......... .......... .......... .......... .......... 17% 39.8M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 28850K .......... .......... .......... .......... .......... 17% 21.0M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 28900K .......... .......... .......... .......... .......... 17% 46.0M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 28950K .......... .......... .......... .......... .......... 17% 25.6M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 29000K .......... .......... .......... .......... .......... 17% 35.4M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 29050K .......... .......... .......... .......... .......... 17% 24.1M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 29100K .......... .......... .......... .......... .......... 17% 24.2M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 29150K .......... .......... .......... .......... .......... 17% 52.6M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 29200K .......... .......... .......... .......... .......... 17% 28.6M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 29250K .......... .......... .......... .......... .......... 17% 25.0M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 29300K .......... .......... .......... .......... .......... 17% 21.7M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 29350K .......... .......... .......... .......... .......... 17% 27.4M 6s 2024-03-29T04:08:41.176 INFO:teuthology.orchestra.run.smithi097.stderr: 29400K .......... .......... .......... .......... .......... 17% 50.4M 6s 2024-03-29T04:08:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 29450K .......... .......... .......... .......... .......... 17% 25.9M 6s 2024-03-29T04:08:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 29500K .......... .......... .......... .......... .......... 17% 31.7M 6s 2024-03-29T04:08:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 29550K .......... .......... .......... .......... .......... 17% 25.0M 6s 2024-03-29T04:08:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 29600K .......... .......... .......... .......... .......... 17% 32.0M 6s 2024-03-29T04:08:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 29650K .......... .......... .......... .......... .......... 17% 33.8M 6s 2024-03-29T04:08:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 29700K .......... .......... .......... .......... .......... 17% 25.0M 6s 2024-03-29T04:08:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 29750K .......... .......... .......... .......... .......... 17% 23.6M 6s 2024-03-29T04:08:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 29800K .......... .......... .......... .......... .......... 17% 60.6M 6s 2024-03-29T04:08:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 29850K .......... .......... .......... .......... .......... 17% 26.7M 6s 2024-03-29T04:08:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 29900K .......... .......... .......... .......... .......... 17% 18.9M 6s 2024-03-29T04:08:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 29950K .......... .......... .......... .......... .......... 17% 35.4M 6s 2024-03-29T04:08:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 30000K .......... .......... .......... .......... .......... 17% 27.0M 6s 2024-03-29T04:08:41.177 INFO:teuthology.orchestra.run.smithi097.stderr: 30050K .......... .......... .......... .......... .......... 17% 37.3M 6s 2024-03-29T04:08:41.178 INFO:teuthology.orchestra.run.smithi097.stderr: 30100K .......... .......... .......... .......... .......... 17% 29.9M 6s 2024-03-29T04:08:41.178 INFO:teuthology.orchestra.run.smithi097.stderr: 30150K .......... .......... .......... .......... .......... 17% 18.2M 6s 2024-03-29T04:08:41.178 INFO:teuthology.orchestra.run.smithi097.stderr: 30200K .......... .......... .......... .......... .......... 17% 49.1M 6s 2024-03-29T04:08:41.178 INFO:teuthology.orchestra.run.smithi097.stderr: 30250K .......... .......... .......... .......... .......... 17% 22.6M 6s 2024-03-29T04:08:41.178 INFO:teuthology.orchestra.run.smithi097.stderr: 30300K .......... .......... .......... .......... .......... 18% 23.5M 6s 2024-03-29T04:08:41.178 INFO:teuthology.orchestra.run.smithi097.stderr: 30350K .......... .......... .......... .......... .......... 18% 23.4M 6s 2024-03-29T04:08:41.178 INFO:teuthology.orchestra.run.smithi097.stderr: 30400K .......... .......... .......... .......... .......... 18% 30.2M 6s 2024-03-29T04:08:41.178 INFO:teuthology.orchestra.run.smithi097.stderr: 30450K .......... .......... .......... .......... .......... 18% 14.3M 6s 2024-03-29T04:08:41.178 INFO:teuthology.orchestra.run.smithi097.stderr: 30500K .......... .......... .......... .......... .......... 18% 21.5M 6s 2024-03-29T04:08:41.178 INFO:teuthology.orchestra.run.smithi097.stderr: 30550K .......... .......... .......... .......... .......... 18% 48.6M 6s 2024-03-29T04:08:41.178 INFO:teuthology.orchestra.run.smithi097.stderr: 30600K .......... .......... .......... .......... .......... 18% 31.2M 6s 2024-03-29T04:08:41.178 INFO:teuthology.orchestra.run.smithi097.stderr: 30650K .......... .......... .......... .......... .......... 18% 4.66M 6s 2024-03-29T04:08:41.178 INFO:teuthology.orchestra.run.smithi097.stderr: 30700K .......... .......... .......... .......... .......... 18% 108M 6s 2024-03-29T04:08:41.179 INFO:teuthology.orchestra.run.smithi097.stderr: 30750K .......... .......... .......... .......... .......... 18% 94.6M 6s 2024-03-29T04:08:41.179 INFO:teuthology.orchestra.run.smithi097.stderr: 30800K .......... .......... .......... .......... .......... 18% 32.7M 6s 2024-03-29T04:08:41.179 INFO:teuthology.orchestra.run.smithi097.stderr: 30850K .......... .......... .......... .......... .......... 18% 112M 6s 2024-03-29T04:08:41.179 INFO:teuthology.orchestra.run.smithi097.stderr: 30900K .......... .......... .......... .......... .......... 18% 8.81M 6s 2024-03-29T04:08:41.179 INFO:teuthology.orchestra.run.smithi097.stderr: 30950K .......... .......... .......... .......... .......... 18% 15.1M 6s 2024-03-29T04:08:41.179 INFO:teuthology.orchestra.run.smithi097.stderr: 31000K .......... .......... .......... .......... .......... 18% 110M 6s 2024-03-29T04:08:41.179 INFO:teuthology.orchestra.run.smithi097.stderr: 31050K .......... .......... .......... .......... .......... 18% 10.6M 6s 2024-03-29T04:08:41.179 INFO:teuthology.orchestra.run.smithi097.stderr: 31100K .......... .......... .......... .......... .......... 18% 11.7M 6s 2024-03-29T04:08:41.179 INFO:teuthology.orchestra.run.smithi097.stderr: 31150K .......... .......... .......... .......... .......... 18% 6.28M 6s 2024-03-29T04:08:41.179 INFO:teuthology.orchestra.run.smithi097.stderr: 31200K .......... .......... .......... .......... .......... 18% 119M 6s 2024-03-29T04:08:41.179 INFO:teuthology.orchestra.run.smithi097.stderr: 31250K .......... .......... .......... .......... .......... 18% 103M 6s 2024-03-29T04:08:41.179 INFO:teuthology.orchestra.run.smithi097.stderr: 31300K .......... .......... .......... .......... .......... 18% 105M 6s 2024-03-29T04:08:41.179 INFO:teuthology.orchestra.run.smithi097.stderr: 31350K .......... .......... .......... .......... .......... 18% 116M 6s 2024-03-29T04:08:47.172 INFO:teuthology.orchestra.run.smithi097.stderr: 31400K .......... .......... .......... .......... .......... 18% 16.6M 6s 2024-03-29T04:08:47.172 INFO:teuthology.orchestra.run.smithi097.stderr: 31450K .......... .......... .......... .......... .......... 18% 50.9M 6s 2024-03-29T04:08:47.172 INFO:teuthology.orchestra.run.smithi097.stderr: 31500K .......... .......... .......... .......... .......... 18% 107M 6s 2024-03-29T04:08:47.172 INFO:teuthology.orchestra.run.smithi097.stderr: 31550K .......... .......... .......... .......... .......... 18% 46.4M 6s 2024-03-29T04:08:47.172 INFO:teuthology.orchestra.run.smithi097.stderr: 31600K .......... .......... .......... .......... .......... 18% 22.9M 6s 2024-03-29T04:08:47.172 INFO:teuthology.orchestra.run.smithi097.stderr: 31650K .......... .......... .......... .......... .......... 18% 27.4M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 31700K .......... .......... .......... .......... .......... 18% 106M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 31750K .......... .......... .......... .......... .......... 18% 21.9M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 31800K .......... .......... .......... .......... .......... 18% 21.9M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 31850K .......... .......... .......... .......... .......... 18% 29.5M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 31900K .......... .......... .......... .......... .......... 18% 115M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 31950K .......... .......... .......... .......... .......... 18% 22.1M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 32000K .......... .......... .......... .......... .......... 19% 22.5M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 32050K .......... .......... .......... .......... .......... 19% 28.1M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 32100K .......... .......... .......... .......... .......... 19% 21.0M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 32150K .......... .......... .......... .......... .......... 19% 113M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 32200K .......... .......... .......... .......... .......... 19% 13.8M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 32250K .......... .......... .......... .......... .......... 19% 27.4M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 32300K .......... .......... .......... .......... .......... 19% 21.5M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 32350K .......... .......... .......... .......... .......... 19% 22.0M 6s 2024-03-29T04:08:47.173 INFO:teuthology.orchestra.run.smithi097.stderr: 32400K .......... .......... .......... .......... .......... 19% 116M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 32450K .......... .......... .......... .......... .......... 19% 22.8M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 32500K .......... .......... .......... .......... .......... 19% 30.2M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 32550K .......... .......... .......... .......... .......... 19% 20.9M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 32600K .......... .......... .......... .......... .......... 19% 115M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 32650K .......... .......... .......... .......... .......... 19% 29.9M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 32700K .......... .......... .......... .......... .......... 19% 22.9M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 32750K .......... .......... .......... .......... .......... 19% 22.3M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 32800K .......... .......... .......... .......... .......... 19% 93.5M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 32850K .......... .......... .......... .......... .......... 19% 22.8M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 32900K .......... .......... .......... .......... .......... 19% 30.7M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 32950K .......... .......... .......... .......... .......... 19% 17.2M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 33000K .......... .......... .......... .......... .......... 19% 21.8M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 33050K .......... .......... .......... .......... .......... 19% 46.4M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 33100K .......... .......... .......... .......... .......... 19% 22.4M 6s 2024-03-29T04:08:47.174 INFO:teuthology.orchestra.run.smithi097.stderr: 33150K .......... .......... .......... .......... .......... 19% 23.7M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33200K .......... .......... .......... .......... .......... 19% 30.5M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33250K .......... .......... .......... .......... .......... 19% 31.3M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33300K .......... .......... .......... .......... .......... 19% 96.5M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33350K .......... .......... .......... .......... .......... 19% 23.3M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33400K .......... .......... .......... .......... .......... 19% 27.5M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33450K .......... .......... .......... .......... .......... 19% 159M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33500K .......... .......... .......... .......... .......... 19% 23.0M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33550K .......... .......... .......... .......... .......... 19% 30.8M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33600K .......... .......... .......... .......... .......... 19% 22.9M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33650K .......... .......... .......... .......... .......... 19% 47.8M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33700K .......... .......... .......... .......... .......... 20% 27.2M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33750K .......... .......... .......... .......... .......... 20% 37.1M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33800K .......... .......... .......... .......... .......... 20% 30.2M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33850K .......... .......... .......... .......... .......... 20% 30.7M 6s 2024-03-29T04:08:47.175 INFO:teuthology.orchestra.run.smithi097.stderr: 33900K .......... .......... .......... .......... .......... 20% 11.2M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 33950K .......... .......... .......... .......... .......... 20% 29.5M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34000K .......... .......... .......... .......... .......... 20% 113M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34050K .......... .......... .......... .......... .......... 20% 23.0M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34100K .......... .......... .......... .......... .......... 20% 21.9M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34150K .......... .......... .......... .......... .......... 20% 27.8M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34200K .......... .......... .......... .......... .......... 20% 21.2M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34250K .......... .......... .......... .......... .......... 20% 117M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34300K .......... .......... .......... .......... .......... 20% 20.4M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34350K .......... .......... .......... .......... .......... 20% 25.8M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34400K .......... .......... .......... .......... .......... 20% 50.3M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34450K .......... .......... .......... .......... .......... 20% 21.7M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34500K .......... .......... .......... .......... .......... 20% 28.3M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34550K .......... .......... .......... .......... .......... 20% 21.8M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34600K .......... .......... .......... .......... .......... 20% 28.6M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34650K .......... .......... .......... .......... .......... 20% 29.7M 6s 2024-03-29T04:08:47.176 INFO:teuthology.orchestra.run.smithi097.stderr: 34700K .......... .......... .......... .......... .......... 20% 22.2M 6s 2024-03-29T04:08:47.177 INFO:teuthology.orchestra.run.smithi097.stderr: 34750K .......... .......... .......... .......... .......... 20% 47.9M 6s 2024-03-29T04:08:47.177 INFO:teuthology.orchestra.run.smithi097.stderr: 34800K .......... .......... .......... .......... .......... 20% 29.9M 6s 2024-03-29T04:08:47.177 INFO:teuthology.orchestra.run.smithi097.stderr: 34850K .......... .......... .......... .......... .......... 20% 28.7M 6s 2024-03-29T04:08:47.177 INFO:teuthology.orchestra.run.smithi097.stderr: 34900K .......... .......... .......... .......... .......... 20% 22.0M 6s 2024-03-29T04:08:47.177 INFO:teuthology.orchestra.run.smithi097.stderr: 34950K .......... .......... .......... .......... .......... 20% 21.1M 6s 2024-03-29T04:08:47.177 INFO:teuthology.orchestra.run.smithi097.stderr: 35000K .......... .......... .......... .......... .......... 20% 121M 6s 2024-03-29T04:08:47.177 INFO:teuthology.orchestra.run.smithi097.stderr: 35050K .......... .......... .......... .......... .......... 20% 22.3M 6s 2024-03-29T04:08:47.177 INFO:teuthology.orchestra.run.smithi097.stderr: 35100K .......... .......... .......... .......... .......... 20% 22.8M 6s 2024-03-29T04:08:47.177 INFO:teuthology.orchestra.run.smithi097.stderr: 35150K .......... .......... .......... .......... .......... 20% 21.0M 6s 2024-03-29T04:08:47.177 INFO:teuthology.orchestra.run.smithi097.stderr: 35200K .......... .......... .......... .......... .......... 20% 36.3M 6s 2024-03-29T04:08:47.177 INFO:teuthology.orchestra.run.smithi097.stderr: 35250K .......... .......... .......... .......... .......... 20% 26.2M 6s 2024-03-29T04:08:47.177 INFO:teuthology.orchestra.run.smithi097.stderr: 35300K .......... .......... .......... .......... .......... 20% 26.7M 6s 2024-03-29T04:08:47.177 INFO:teuthology.orchestra.run.smithi097.stderr: 35350K .......... .......... .......... .......... .......... 21% 30.4M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 35400K .......... .......... .......... .......... .......... 21% 30.9M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 35450K .......... .......... .......... .......... .......... 21% 20.8M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 35500K .......... .......... .......... .......... .......... 21% 32.5M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 35550K .......... .......... .......... .......... .......... 21% 22.0M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 35600K .......... .......... .......... .......... .......... 21% 49.4M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 35650K .......... .......... .......... .......... .......... 21% 21.8M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 35700K .......... .......... .......... .......... .......... 21% 25.7M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 35750K .......... .......... .......... .......... .......... 21% 38.2M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 35800K .......... .......... .......... .......... .......... 21% 29.6M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 35850K .......... .......... .......... .......... .......... 21% 29.2M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 35900K .......... .......... .......... .......... .......... 21% 28.3M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 35950K .......... .......... .......... .......... .......... 21% 22.8M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 36000K .......... .......... .......... .......... .......... 21% 31.5M 6s 2024-03-29T04:08:47.178 INFO:teuthology.orchestra.run.smithi097.stderr: 36050K .......... .......... .......... .......... .......... 21% 27.0M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36100K .......... .......... .......... .......... .......... 21% 36.8M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36150K .......... .......... .......... .......... .......... 21% 47.5M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36200K .......... .......... .......... .......... .......... 21% 22.4M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36250K .......... .......... .......... .......... .......... 21% 64.6M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36300K .......... .......... .......... .......... .......... 21% 57.5M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36350K .......... .......... .......... .......... .......... 21% 103M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36400K .......... .......... .......... .......... .......... 21% 102M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36450K .......... .......... .......... .......... .......... 21% 96.0M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36500K .......... .......... .......... .......... .......... 21% 102M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36550K .......... .......... .......... .......... .......... 21% 99.8M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36600K .......... .......... .......... .......... .......... 21% 99.0M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36650K .......... .......... .......... .......... .......... 21% 96.7M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36700K .......... .......... .......... .......... .......... 21% 93.6M 6s 2024-03-29T04:08:47.179 INFO:teuthology.orchestra.run.smithi097.stderr: 36750K .......... .......... .......... .......... .......... 21% 112M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 36800K .......... .......... .......... .......... .......... 21% 90.1M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 36850K .......... .......... .......... .......... .......... 21% 103M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 36900K .......... .......... .......... .......... .......... 21% 92.9M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 36950K .......... .......... .......... .......... .......... 21% 95.9M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 37000K .......... .......... .......... .......... .......... 21% 109M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 37050K .......... .......... .......... .......... .......... 22% 67.9M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 37100K .......... .......... .......... .......... .......... 22% 43.6M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 37150K .......... .......... .......... .......... .......... 22% 27.1M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 37200K .......... .......... .......... .......... .......... 22% 37.0M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 37250K .......... .......... .......... .......... .......... 22% 35.8M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 37300K .......... .......... .......... .......... .......... 22% 29.0M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 37350K .......... .......... .......... .......... .......... 22% 33.9M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 37400K .......... .......... .......... .......... .......... 22% 22.3M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 37450K .......... .......... .......... .......... .......... 22% 40.5M 6s 2024-03-29T04:08:47.180 INFO:teuthology.orchestra.run.smithi097.stderr: 37500K .......... .......... .......... .......... .......... 22% 32.5M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 37550K .......... .......... .......... .......... .......... 22% 22.9M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 37600K .......... .......... .......... .......... .......... 22% 25.5M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 37650K .......... .......... .......... .......... .......... 22% 54.1M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 37700K .......... .......... .......... .......... .......... 22% 26.4M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 37750K .......... .......... .......... .......... .......... 22% 23.9M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 37800K .......... .......... .......... .......... .......... 22% 41.3M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 37850K .......... .......... .......... .......... .......... 22% 23.7M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 37900K .......... .......... .......... .......... .......... 22% 37.1M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 37950K .......... .......... .......... .......... .......... 22% 24.3M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 38000K .......... .......... .......... .......... .......... 22% 25.6M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 38050K .......... .......... .......... .......... .......... 22% 34.9M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 38100K .......... .......... .......... .......... .......... 22% 24.7M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 38150K .......... .......... .......... .......... .......... 22% 54.5M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 38200K .......... .......... .......... .......... .......... 22% 23.1M 6s 2024-03-29T04:08:47.181 INFO:teuthology.orchestra.run.smithi097.stderr: 38250K .......... .......... .......... .......... .......... 22% 24.1M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38300K .......... .......... .......... .......... .......... 22% 35.4M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38350K .......... .......... .......... .......... .......... 22% 25.3M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38400K .......... .......... .......... .......... .......... 22% 52.8M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38450K .......... .......... .......... .......... .......... 22% 24.7M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38500K .......... .......... .......... .......... .......... 22% 23.4M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38550K .......... .......... .......... .......... .......... 22% 35.3M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38600K .......... .......... .......... .......... .......... 22% 23.3M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38650K .......... .......... .......... .......... .......... 22% 24.9M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38700K .......... .......... .......... .......... .......... 22% 45.2M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38750K .......... .......... .......... .......... .......... 23% 24.5M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38800K .......... .......... .......... .......... .......... 23% 35.6M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38850K .......... .......... .......... .......... .......... 23% 24.2M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38900K .......... .......... .......... .......... .......... 23% 36.4M 6s 2024-03-29T04:08:47.182 INFO:teuthology.orchestra.run.smithi097.stderr: 38950K .......... .......... .......... .......... .......... 23% 27.8M 6s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39000K .......... .......... .......... .......... .......... 23% 33.1M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39050K .......... .......... .......... .......... .......... 23% 22.9M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39100K .......... .......... .......... .......... .......... 23% 26.1M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39150K .......... .......... .......... .......... .......... 23% 37.8M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39200K .......... .......... .......... .......... .......... 23% 25.6M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39250K .......... .......... .......... .......... .......... 23% 43.2M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39300K .......... .......... .......... .......... .......... 23% 21.5M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39350K .......... .......... .......... .......... .......... 23% 28.4M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39400K .......... .......... .......... .......... .......... 23% 35.1M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39450K .......... .......... .......... .......... .......... 23% 24.9M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39500K .......... .......... .......... .......... .......... 23% 24.2M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39550K .......... .......... .......... .......... .......... 23% 52.5M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39600K .......... .......... .......... .......... .......... 23% 24.9M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39650K .......... .......... .......... .......... .......... 23% 31.5M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39700K .......... .......... .......... .......... .......... 23% 24.2M 5s 2024-03-29T04:08:47.183 INFO:teuthology.orchestra.run.smithi097.stderr: 39750K .......... .......... .......... .......... .......... 23% 24.1M 5s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 39800K .......... .......... .......... .......... .......... 23% 61.4M 5s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 39850K .......... .......... .......... .......... .......... 23% 30.1M 5s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 39900K .......... .......... .......... .......... .......... 23% 497K 6s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 39950K .......... .......... .......... .......... .......... 23% 57.0M 6s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 40000K .......... .......... .......... .......... .......... 23% 80.5M 6s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 40050K .......... .......... .......... .......... .......... 23% 32.5M 6s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 40100K .......... .......... .......... .......... .......... 23% 416M 6s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 40150K .......... .......... .......... .......... .......... 23% 72.8M 6s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 40200K .......... .......... .......... .......... .......... 23% 718M 6s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 40250K .......... .......... .......... .......... .......... 23% 63.4M 6s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 40300K .......... .......... .......... .......... .......... 23% 633M 6s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 40350K .......... .......... .......... .......... .......... 23% 714M 6s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 40400K .......... .......... .......... .......... .......... 23% 598M 6s 2024-03-29T04:08:47.184 INFO:teuthology.orchestra.run.smithi097.stderr: 40450K .......... .......... .......... .......... .......... 24% 46.3M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 40500K .......... .......... .......... .......... .......... 24% 156M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 40550K .......... .......... .......... .......... .......... 24% 714M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 40600K .......... .......... .......... .......... .......... 24% 645M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 40650K .......... .......... .......... .......... .......... 24% 714M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 40700K .......... .......... .......... .......... .......... 24% 732M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 40750K .......... .......... .......... .......... .......... 24% 722M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 40800K .......... .......... .......... .......... .......... 24% 636M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 40850K .......... .......... .......... .......... .......... 24% 690M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 40900K .......... .......... .......... .......... .......... 24% 728M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 40950K .......... .......... .......... .......... .......... 24% 523M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 41000K .......... .......... .......... .......... .......... 24% 289M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 41050K .......... .......... .......... .......... .......... 24% 661M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 41100K .......... .......... .......... .......... .......... 24% 642M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 41150K .......... .......... .......... .......... .......... 24% 723M 6s 2024-03-29T04:08:47.185 INFO:teuthology.orchestra.run.smithi097.stderr: 41200K .......... .......... .......... .......... .......... 24% 691M 6s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41250K .......... .......... .......... .......... .......... 24% 720M 6s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41300K .......... .......... .......... .......... .......... 24% 612M 6s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41350K .......... .......... .......... .......... .......... 24% 703M 6s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41400K .......... .......... .......... .......... .......... 24% 696M 6s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41450K .......... .......... .......... .......... .......... 24% 635M 5s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41500K .......... .......... .......... .......... .......... 24% 598M 5s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41550K .......... .......... .......... .......... .......... 24% 697M 5s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41600K .......... .......... .......... .......... .......... 24% 714M 5s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41650K .......... .......... .......... .......... .......... 24% 702M 5s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41700K .......... .......... .......... .......... .......... 24% 588M 5s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41750K .......... .......... .......... .......... .......... 24% 697M 5s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41800K .......... .......... .......... .......... .......... 24% 705M 5s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41850K .......... .......... .......... .......... .......... 24% 701M 5s 2024-03-29T04:08:47.186 INFO:teuthology.orchestra.run.smithi097.stderr: 41900K .......... .......... .......... .......... .......... 24% 609M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 41950K .......... .......... .......... .......... .......... 24% 677M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 42000K .......... .......... .......... .......... .......... 24% 715M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 42050K .......... .......... .......... .......... .......... 24% 638M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 42100K .......... .......... .......... .......... .......... 25% 27.2M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 42150K .......... .......... .......... .......... .......... 25% 32.7M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 42200K .......... .......... .......... .......... .......... 25% 681M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 42250K .......... .......... .......... .......... .......... 25% 718M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 42300K .......... .......... .......... .......... .......... 25% 717M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 42350K .......... .......... .......... .......... .......... 25% 614M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 42400K .......... .......... .......... .......... .......... 25% 713M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 42450K .......... .......... .......... .......... .......... 25% 689M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 42500K .......... .......... .......... .......... .......... 25% 4.13M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 42550K .......... .......... .......... .......... .......... 25% 635M 5s 2024-03-29T04:08:47.187 INFO:teuthology.orchestra.run.smithi097.stderr: 42600K .......... .......... .......... .......... .......... 25% 613M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 42650K .......... .......... .......... .......... .......... 25% 692M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 42700K .......... .......... .......... .......... .......... 25% 673M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 42750K .......... .......... .......... .......... .......... 25% 703M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 42800K .......... .......... .......... .......... .......... 25% 627M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 42850K .......... .......... .......... .......... .......... 25% 733M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 42900K .......... .......... .......... .......... .......... 25% 660M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 42950K .......... .......... .......... .......... .......... 25% 697M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 43000K .......... .......... .......... .......... .......... 25% 586M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 43050K .......... .......... .......... .......... .......... 25% 701M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 43100K .......... .......... .......... .......... .......... 25% 736M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 43150K .......... .......... .......... .......... .......... 25% 2.93M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 43200K .......... .......... .......... .......... .......... 25% 116M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 43250K .......... .......... .......... .......... .......... 25% 113M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 43300K .......... .......... .......... .......... .......... 25% 111M 5s 2024-03-29T04:08:47.188 INFO:teuthology.orchestra.run.smithi097.stderr: 43350K .......... .......... .......... .......... .......... 25% 1.72M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 43400K .......... .......... .......... .......... .......... 25% 115M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 43450K .......... .......... .......... .......... .......... 25% 109M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 43500K .......... .......... .......... .......... .......... 25% 37.2M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 43550K .......... .......... .......... .......... .......... 25% 85.4M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 43600K .......... .......... .......... .......... .......... 25% 43.4M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 43650K .......... .......... .......... .......... .......... 25% 44.7M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 43700K .......... .......... .......... .......... .......... 25% 94.4M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 43750K .......... .......... .......... .......... .......... 25% 48.9M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 43800K .......... .......... .......... .......... .......... 26% 90.0M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 43850K .......... .......... .......... .......... .......... 26% 73.3M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 43900K .......... .......... .......... .......... .......... 26% 33.2M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 43950K .......... .......... .......... .......... .......... 26% 116M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 44000K .......... .......... .......... .......... .......... 26% 22.7M 5s 2024-03-29T04:08:47.189 INFO:teuthology.orchestra.run.smithi097.stderr: 44050K .......... .......... .......... .......... .......... 26% 25.4M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44100K .......... .......... .......... .......... .......... 26% 22.2M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44150K .......... .......... .......... .......... .......... 26% 116M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44200K .......... .......... .......... .......... .......... 26% 28.9M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44250K .......... .......... .......... .......... .......... 26% 22.2M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44300K .......... .......... .......... .......... .......... 26% 27.3M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44350K .......... .......... .......... .......... .......... 26% 67.7M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44400K .......... .......... .......... .......... .......... 26% 16.7M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44450K .......... .......... .......... .......... .......... 26% 41.5M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44500K .......... .......... .......... .......... .......... 26% 24.8M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44550K .......... .......... .......... .......... .......... 26% 28.3M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44600K .......... .......... .......... .......... .......... 26% 68.7M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44650K .......... .......... .......... .......... .......... 26% 21.5M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44700K .......... .......... .......... .......... .......... 26% 26.3M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44750K .......... .......... .......... .......... .......... 26% 17.3M 5s 2024-03-29T04:08:47.190 INFO:teuthology.orchestra.run.smithi097.stderr: 44800K .......... .......... .......... .......... .......... 26% 38.1M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 44850K .......... .......... .......... .......... .......... 26% 87.7M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 44900K .......... .......... .......... .......... .......... 26% 25.0M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 44950K .......... .......... .......... .......... .......... 26% 27.6M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 45000K .......... .......... .......... .......... .......... 26% 21.2M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 45050K .......... .......... .......... .......... .......... 26% 27.3M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 45100K .......... .......... .......... .......... .......... 26% 116M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 45150K .......... .......... .......... .......... .......... 26% 22.4M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 45200K .......... .......... .......... .......... .......... 26% 31.1M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 45250K .......... .......... .......... .......... .......... 26% 32.8M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 45300K .......... .......... .......... .......... .......... 26% 26.8M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 45350K .......... .......... .......... .......... .......... 26% 20.1M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 45400K .......... .......... .......... .......... .......... 26% 63.0M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 45450K .......... .......... .......... .......... .......... 26% 24.4M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 45500K .......... .......... .......... .......... .......... 27% 24.7M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 45550K .......... .......... .......... .......... .......... 27% 25.1M 5s 2024-03-29T04:08:47.191 INFO:teuthology.orchestra.run.smithi097.stderr: 45600K .......... .......... .......... .......... .......... 27% 24.5M 5s 2024-03-29T04:08:47.192 INFO:teuthology.orchestra.run.smithi097.stderr: 45650K .......... .......... .......... .......... .......... 27% 63.0M 5s 2024-03-29T04:08:47.192 INFO:teuthology.orchestra.run.smithi097.stderr: 45700K .......... .......... .......... .......... .......... 27% 27.0M 5s 2024-03-29T04:08:47.192 INFO:teuthology.orchestra.run.smithi097.stderr: 45750K .......... .......... .......... .......... .......... 27% 25.0M 5s 2024-03-29T04:08:47.192 INFO:teuthology.orchestra.run.smithi097.stderr: 45800K .......... .......... .......... .......... .......... 27% 23.8M 5s 2024-03-29T04:08:47.192 INFO:teuthology.orchestra.run.smithi097.stderr: 45850K .......... .......... .......... .......... .......... 27% 57.9M 5s 2024-03-29T04:08:47.192 INFO:teuthology.orchestra.run.smithi097.stderr: 45900K .......... .......... .......... .......... .......... 27% 29.7M 5s 2024-03-29T04:08:47.192 INFO:teuthology.orchestra.run.smithi097.stderr: 45950K .......... .......... .......... .......... .......... 27% 23.4M 5s 2024-03-29T04:08:47.192 INFO:teuthology.orchestra.run.smithi097.stderr: 46000K .......... .......... .......... .......... .......... 27% 24.1M 5s 2024-03-29T04:08:47.192 INFO:teuthology.orchestra.run.smithi097.stderr: 46050K .......... .......... .......... .......... .......... 27% 35.6M 5s 2024-03-29T04:08:47.192 INFO:teuthology.orchestra.run.smithi097.stderr: 46100K .......... .......... .......... .......... .......... 27% 20.1M 5s 2024-03-29T04:08:47.192 INFO:teuthology.orchestra.run.smithi097.stderr: 46150K .......... .......... .......... .......... .......... 27% 46.3M 5s 2024-03-29T04:08:47.192 INFO:teuthology.orchestra.run.smithi097.stderr: 46200K .......... .......... .......... .......... .......... 27% 23.9M 5s 2024-03-29T04:08:47.192 INFO:teuthology.orchestra.run.smithi097.stderr: 46250K .......... .......... .......... .......... .......... 27% 34.2M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46300K .......... .......... .......... .......... .......... 27% 27.9M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46350K .......... .......... .......... .......... .......... 27% 20.4M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46400K .......... .......... .......... .......... .......... 27% 70.4M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46450K .......... .......... .......... .......... .......... 27% 19.8M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46500K .......... .......... .......... .......... .......... 27% 33.2M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46550K .......... .......... .......... .......... .......... 27% 26.1M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46600K .......... .......... .......... .......... .......... 27% 24.9M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46650K .......... .......... .......... .......... .......... 27% 73.8M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46700K .......... .......... .......... .......... .......... 27% 24.0M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46750K .......... .......... .......... .......... .......... 27% 26.3M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46800K .......... .......... .......... .......... .......... 27% 25.8M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46850K .......... .......... .......... .......... .......... 27% 25.0M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46900K .......... .......... .......... .......... .......... 27% 58.4M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 46950K .......... .......... .......... .......... .......... 27% 31.6M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 47000K .......... .......... .......... .......... .......... 27% 25.7M 5s 2024-03-29T04:08:47.193 INFO:teuthology.orchestra.run.smithi097.stderr: 47050K .......... .......... .......... .......... .......... 27% 18.9M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47100K .......... .......... .......... .......... .......... 27% 36.4M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47150K .......... .......... .......... .......... .......... 28% 52.8M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47200K .......... .......... .......... .......... .......... 28% 25.0M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47250K .......... .......... .......... .......... .......... 28% 32.9M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47300K .......... .......... .......... .......... .......... 28% 25.1M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47350K .......... .......... .......... .......... .......... 28% 25.2M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47400K .......... .......... .......... .......... .......... 28% 47.6M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47450K .......... .......... .......... .......... .......... 28% 10.9M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47500K .......... .......... .......... .......... .......... 28% 391M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47550K .......... .......... .......... .......... .......... 28% 59.9M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47600K .......... .......... .......... .......... .......... 28% 23.9M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47650K .......... .......... .......... .......... .......... 28% 50.6M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47700K .......... .......... .......... .......... .......... 28% 24.8M 5s 2024-03-29T04:08:47.194 INFO:teuthology.orchestra.run.smithi097.stderr: 47750K .......... .......... .......... .......... .......... 28% 24.0M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 47800K .......... .......... .......... .......... .......... 28% 31.1M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 47850K .......... .......... .......... .......... .......... 28% 25.2M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 47900K .......... .......... .......... .......... .......... 28% 68.9M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 47950K .......... .......... .......... .......... .......... 28% 23.0M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 48000K .......... .......... .......... .......... .......... 28% 29.4M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 48050K .......... .......... .......... .......... .......... 28% 27.9M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 48100K .......... .......... .......... .......... .......... 28% 31.5M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 48150K .......... .......... .......... .......... .......... 28% 38.5M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 48200K .......... .......... .......... .......... .......... 28% 27.1M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 48250K .......... .......... .......... .......... .......... 28% 24.8M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 48300K .......... .......... .......... .......... .......... 28% 27.7M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 48350K .......... .......... .......... .......... .......... 28% 24.7M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 48400K .......... .......... .......... .......... .......... 28% 24.7M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 48450K .......... .......... .......... .......... .......... 28% 57.2M 5s 2024-03-29T04:08:47.195 INFO:teuthology.orchestra.run.smithi097.stderr: 48500K .......... .......... .......... .......... .......... 28% 25.7M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 48550K .......... .......... .......... .......... .......... 28% 30.7M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 48600K .......... .......... .......... .......... .......... 28% 24.7M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 48650K .......... .......... .......... .......... .......... 28% 24.1M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 48700K .......... .......... .......... .......... .......... 28% 49.5M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 48750K .......... .......... .......... .......... .......... 28% 26.1M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 48800K .......... .......... .......... .......... .......... 28% 30.9M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 48850K .......... .......... .......... .......... .......... 29% 34.3M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 48900K .......... .......... .......... .......... .......... 29% 16.1M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 48950K .......... .......... .......... .......... .......... 29% 52.8M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 49000K .......... .......... .......... .......... .......... 29% 24.6M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 49050K .......... .......... .......... .......... .......... 29% 32.3M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 49100K .......... .......... .......... .......... .......... 29% 25.5M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 49150K .......... .......... .......... .......... .......... 29% 16.8M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 49200K .......... .......... .......... .......... .......... 29% 58.6M 5s 2024-03-29T04:08:47.196 INFO:teuthology.orchestra.run.smithi097.stderr: 49250K .......... .......... .......... .......... .......... 29% 38.2M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49300K .......... .......... .......... .......... .......... 29% 33.6M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49350K .......... .......... .......... .......... .......... 29% 24.1M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49400K .......... .......... .......... .......... .......... 29% 24.8M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49450K .......... .......... .......... .......... .......... 29% 44.9M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49500K .......... .......... .......... .......... .......... 29% 33.0M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49550K .......... .......... .......... .......... .......... 29% 26.5M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49600K .......... .......... .......... .......... .......... 29% 23.8M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49650K .......... .......... .......... .......... .......... 29% 40.4M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49700K .......... .......... .......... .......... .......... 29% 28.9M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49750K .......... .......... .......... .......... .......... 29% 22.8M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49800K .......... .......... .......... .......... .......... 29% 26.1M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49850K .......... .......... .......... .......... .......... 29% 28.9M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49900K .......... .......... .......... .......... .......... 29% 29.4M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 49950K .......... .......... .......... .......... .......... 29% 38.6M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 50000K .......... .......... .......... .......... .......... 29% 30.9M 5s 2024-03-29T04:08:47.197 INFO:teuthology.orchestra.run.smithi097.stderr: 50050K .......... .......... .......... .......... .......... 29% 24.3M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50100K .......... .......... .......... .......... .......... 29% 24.6M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50150K .......... .......... .......... .......... .......... 29% 58.5M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50200K .......... .......... .......... .......... .......... 29% 20.1M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50250K .......... .......... .......... .......... .......... 29% 25.2M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50300K .......... .......... .......... .......... .......... 29% 44.6M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50350K .......... .......... .......... .......... .......... 29% 23.8M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50400K .......... .......... .......... .......... .......... 29% 55.9M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50450K .......... .......... .......... .......... .......... 29% 27.1M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50500K .......... .......... .......... .......... .......... 29% 27.6M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50550K .......... .......... .......... .......... .......... 30% 24.4M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50600K .......... .......... .......... .......... .......... 30% 23.9M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50650K .......... .......... .......... .......... .......... 30% 35.8M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50700K .......... .......... .......... .......... .......... 30% 35.7M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50750K .......... .......... .......... .......... .......... 30% 28.9M 5s 2024-03-29T04:08:47.198 INFO:teuthology.orchestra.run.smithi097.stderr: 50800K .......... .......... .......... .......... .......... 30% 24.3M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 50850K .......... .......... .......... .......... .......... 30% 23.3M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 50900K .......... .......... .......... .......... .......... 30% 50.4M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 50950K .......... .......... .......... .......... .......... 30% 33.5M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 51000K .......... .......... .......... .......... .......... 30% 26.0M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 51050K .......... .......... .......... .......... .......... 30% 16.3M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 51100K .......... .......... .......... .......... .......... 30% 95.6M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 51150K .......... .......... .......... .......... .......... 30% 29.1M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 51200K .......... .......... .......... .......... .......... 30% 34.4M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 51250K .......... .......... .......... .......... .......... 30% 25.3M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 51300K .......... .......... .......... .......... .......... 30% 23.4M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 51350K .......... .......... .......... .......... .......... 30% 52.4M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 51400K .......... .......... .......... .......... .......... 30% 24.6M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 51450K .......... .......... .......... .......... .......... 30% 35.0M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 51500K .......... .......... .......... .......... .......... 30% 23.4M 5s 2024-03-29T04:08:47.199 INFO:teuthology.orchestra.run.smithi097.stderr: 51550K .......... .......... .......... .......... .......... 30% 23.2M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 51600K .......... .......... .......... .......... .......... 30% 37.8M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 51650K .......... .......... .......... .......... .......... 30% 27.3M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 51700K .......... .......... .......... .......... .......... 30% 36.3M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 51750K .......... .......... .......... .......... .......... 30% 20.0M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 51800K .......... .......... .......... .......... .......... 30% 58.8M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 51850K .......... .......... .......... .......... .......... 30% 24.6M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 51900K .......... .......... .......... .......... .......... 30% 24.5M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 51950K .......... .......... .......... .......... .......... 30% 44.8M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 52000K .......... .......... .......... .......... .......... 30% 24.7M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 52050K .......... .......... .......... .......... .......... 30% 38.6M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 52100K .......... .......... .......... .......... .......... 30% 24.7M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 52150K .......... .......... .......... .......... .......... 30% 24.0M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 52200K .......... .......... .......... .......... .......... 30% 47.2M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 52250K .......... .......... .......... .......... .......... 31% 23.9M 5s 2024-03-29T04:08:47.200 INFO:teuthology.orchestra.run.smithi097.stderr: 52300K .......... .......... .......... .......... .......... 31% 35.6M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 52350K .......... .......... .......... .......... .......... 31% 24.0M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 52400K .......... .......... .......... .......... .......... 31% 23.7M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 52450K .......... .......... .......... .......... .......... 31% 58.4M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 52500K .......... .......... .......... .......... .......... 31% 19.6M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 52550K .......... .......... .......... .......... .......... 31% 32.4M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 52600K .......... .......... .......... .......... .......... 31% 24.6M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 52650K .......... .......... .......... .......... .......... 31% 25.1M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 52700K .......... .......... .......... .......... .......... 31% 51.5M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 52750K .......... .......... .......... .......... .......... 31% 28.4M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 52800K .......... .......... .......... .......... .......... 31% 24.6M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 52850K .......... .......... .......... .......... .......... 31% 15.7M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 52900K .......... .......... .......... .......... .......... 31% 35.0M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 52950K .......... .......... .......... .......... .......... 31% 95.7M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 53000K .......... .......... .......... .......... .......... 31% 30.9M 5s 2024-03-29T04:08:47.201 INFO:teuthology.orchestra.run.smithi097.stderr: 53050K .......... .......... .......... .......... .......... 31% 25.4M 5s 2024-03-29T04:08:47.202 INFO:teuthology.orchestra.run.smithi097.stderr: 53100K .......... .......... .......... .......... .......... 31% 30.3M 5s 2024-03-29T04:08:47.202 INFO:teuthology.orchestra.run.smithi097.stderr: 53150K .......... .......... .......... .......... .......... 31% 24.3M 5s 2024-03-29T04:08:47.202 INFO:teuthology.orchestra.run.smithi097.stderr: 53200K .......... .......... .......... .......... .......... 31% 49.6M 5s 2024-03-29T04:08:47.202 INFO:teuthology.orchestra.run.smithi097.stderr: 53250K .......... .......... .......... .......... .......... 31% 23.5M 5s 2024-03-29T04:08:47.202 INFO:teuthology.orchestra.run.smithi097.stderr: 53300K .......... .......... .......... .......... .......... 31% 33.7M 5s 2024-03-29T04:08:47.202 INFO:teuthology.orchestra.run.smithi097.stderr: 53350K .......... .......... .......... .......... .......... 31% 25.0M 5s 2024-03-29T04:08:47.202 INFO:teuthology.orchestra.run.smithi097.stderr: 53400K .......... .......... .......... .......... .......... 31% 38.3M 5s 2024-03-29T04:08:47.202 INFO:teuthology.orchestra.run.smithi097.stderr: 53450K .......... .......... .......... .......... .......... 31% 28.4M 5s 2024-03-29T04:08:47.202 INFO:teuthology.orchestra.run.smithi097.stderr: 53500K .......... .......... .......... .......... .......... 31% 27.5M 5s 2024-03-29T04:08:47.202 INFO:teuthology.orchestra.run.smithi097.stderr: 53550K .......... .......... .......... .......... .......... 31% 23.6M 5s 2024-03-29T04:08:47.202 INFO:teuthology.orchestra.run.smithi097.stderr: 53600K .......... .......... .......... .......... .......... 31% 28.3M 5s 2024-03-29T04:08:47.202 INFO:teuthology.orchestra.run.smithi097.stderr: 53650K .......... .......... .......... .......... .......... 31% 20.1M 5s 2024-03-29T04:08:47.202 INFO:teuthology.orchestra.run.smithi097.stderr: 53700K .......... .......... .......... .......... .......... 31% 24.2M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 53750K .......... .......... .......... .......... .......... 31% 115M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 53800K .......... .......... .......... .......... .......... 31% 35.2M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 53850K .......... .......... .......... .......... .......... 31% 25.2M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 53900K .......... .......... .......... .......... .......... 32% 24.4M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 53950K .......... .......... .......... .......... .......... 32% 24.1M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 54000K .......... .......... .......... .......... .......... 32% 67.6M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 54050K .......... .......... .......... .......... .......... 32% 28.0M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 54100K .......... .......... .......... .......... .......... 32% 26.2M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 54150K .......... .......... .......... .......... .......... 32% 24.6M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 54200K .......... .......... .......... .......... .......... 32% 32.3M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 54250K .......... .......... .......... .......... .......... 32% 35.0M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 54300K .......... .......... .......... .......... .......... 32% 29.9M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 54350K .......... .......... .......... .......... .......... 32% 30.4M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 54400K .......... .......... .......... .......... .......... 32% 22.5M 5s 2024-03-29T04:08:47.203 INFO:teuthology.orchestra.run.smithi097.stderr: 54450K .......... .......... .......... .......... .......... 32% 24.4M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 54500K .......... .......... .......... .......... .......... 32% 53.9M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 54550K .......... .......... .......... .......... .......... 32% 24.7M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 54600K .......... .......... .......... .......... .......... 32% 31.4M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 54650K .......... .......... .......... .......... .......... 32% 24.5M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 54700K .......... .......... .......... .......... .......... 32% 30.3M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 54750K .......... .......... .......... .......... .......... 32% 35.8M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 54800K .......... .......... .......... .......... .......... 32% 28.0M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 54850K .......... .......... .......... .......... .......... 32% 30.6M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 54900K .......... .......... .......... .......... .......... 32% 24.2M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 54950K .......... .......... .......... .......... .......... 32% 24.7M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 55000K .......... .......... .......... .......... .......... 32% 49.1M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 55050K .......... .......... .......... .......... .......... 32% 28.1M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 55100K .......... .......... .......... .......... .......... 32% 29.7M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 55150K .......... .......... .......... .......... .......... 32% 25.1M 5s 2024-03-29T04:08:47.204 INFO:teuthology.orchestra.run.smithi097.stderr: 55200K .......... .......... .......... .......... .......... 32% 24.6M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55250K .......... .......... .......... .......... .......... 32% 44.3M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55300K .......... .......... .......... .......... .......... 32% 25.1M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55350K .......... .......... .......... .......... .......... 32% 35.2M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55400K .......... .......... .......... .......... .......... 32% 16.4M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55450K .......... .......... .......... .......... .......... 32% 42.4M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55500K .......... .......... .......... .......... .......... 32% 69.8M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55550K .......... .......... .......... .......... .......... 32% 23.4M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55600K .......... .......... .......... .......... .......... 33% 34.2M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55650K .......... .......... .......... .......... .......... 33% 24.0M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55700K .......... .......... .......... .......... .......... 33% 24.2M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55750K .......... .......... .......... .......... .......... 33% 43.9M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55800K .......... .......... .......... .......... .......... 33% 24.6M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55850K .......... .......... .......... .......... .......... 33% 36.6M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55900K .......... .......... .......... .......... .......... 33% 25.1M 5s 2024-03-29T04:08:47.205 INFO:teuthology.orchestra.run.smithi097.stderr: 55950K .......... .......... .......... .......... .......... 33% 24.6M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56000K .......... .......... .......... .......... .......... 33% 44.6M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56050K .......... .......... .......... .......... .......... 33% 23.8M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56100K .......... .......... .......... .......... .......... 33% 24.3M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56150K .......... .......... .......... .......... .......... 33% 34.1M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56200K .......... .......... .......... .......... .......... 33% 44.7M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56250K .......... .......... .......... .......... .......... 33% 26.4M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56300K .......... .......... .......... .......... .......... 33% 30.4M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56350K .......... .......... .......... .......... .......... 33% 25.5M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56400K .......... .......... .......... .......... .......... 33% 24.5M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56450K .......... .......... .......... .......... .......... 33% 29.4M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56500K .......... .......... .......... .......... .......... 33% 36.0M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56550K .......... .......... .......... .......... .......... 33% 26.5M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56600K .......... .......... .......... .......... .......... 33% 31.9M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56650K .......... .......... .......... .......... .......... 33% 20.3M 5s 2024-03-29T04:08:47.206 INFO:teuthology.orchestra.run.smithi097.stderr: 56700K .......... .......... .......... .......... .......... 33% 59.5M 5s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 56750K .......... .......... .......... .......... .......... 33% 28.9M 5s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 56800K .......... .......... .......... .......... .......... 33% 26.8M 5s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 56850K .......... .......... .......... .......... .......... 33% 24.9M 5s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 56900K .......... .......... .......... .......... .......... 33% 18.8M 5s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 56950K .......... .......... .......... .......... .......... 33% 92.9M 5s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 57000K .......... .......... .......... .......... .......... 33% 33.4M 5s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 57050K .......... .......... .......... .......... .......... 33% 23.9M 5s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 57100K .......... .......... .......... .......... .......... 33% 27.8M 5s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 57150K .......... .......... .......... .......... .......... 33% 22.5M 5s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 57200K .......... .......... .......... .......... .......... 33% 54.6M 5s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 57250K .......... .......... .......... .......... .......... 33% 32.9M 5s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 57300K .......... .......... .......... .......... .......... 34% 21.3M 4s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 57350K .......... .......... .......... .......... .......... 34% 37.1M 4s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 57400K .......... .......... .......... .......... .......... 34% 19.6M 4s 2024-03-29T04:08:47.207 INFO:teuthology.orchestra.run.smithi097.stderr: 57450K .......... .......... .......... .......... .......... 34% 53.6M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 57500K .......... .......... .......... .......... .......... 34% 31.0M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 57550K .......... .......... .......... .......... .......... 34% 25.4M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 57600K .......... .......... .......... .......... .......... 34% 25.1M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 57650K .......... .......... .......... .......... .......... 34% 24.7M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 57700K .......... .......... .......... .......... .......... 34% 46.8M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 57750K .......... .......... .......... .......... .......... 34% 32.5M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 57800K .......... .......... .......... .......... .......... 34% 24.2M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 57850K .......... .......... .......... .......... .......... 34% 23.6M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 57900K .......... .......... .......... .......... .......... 34% 49.5M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 57950K .......... .......... .......... .......... .......... 34% 20.4M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 58000K .......... .......... .......... .......... .......... 34% 54.7M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 58050K .......... .......... .......... .......... .......... 34% 22.3M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 58100K .......... .......... .......... .......... .......... 34% 24.6M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 58150K .......... .......... .......... .......... .......... 34% 36.6M 4s 2024-03-29T04:08:47.208 INFO:teuthology.orchestra.run.smithi097.stderr: 58200K .......... .......... .......... .......... .......... 34% 25.3M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58250K .......... .......... .......... .......... .......... 34% 37.7M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58300K .......... .......... .......... .......... .......... 34% 27.0M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58350K .......... .......... .......... .......... .......... 34% 28.9M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58400K .......... .......... .......... .......... .......... 34% 30.5M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58450K .......... .......... .......... .......... .......... 34% 28.5M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58500K .......... .......... .......... .......... .......... 34% 36.4M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58550K .......... .......... .......... .......... .......... 34% 25.0M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58600K .......... .......... .......... .......... .......... 34% 32.7M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58650K .......... .......... .......... .......... .......... 34% 17.0M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58700K .......... .......... .......... .......... .......... 34% 47.4M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58750K .......... .......... .......... .......... .......... 34% 24.4M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58800K .......... .......... .......... .......... .......... 34% 43.5M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58850K .......... .......... .......... .......... .......... 34% 39.2M 4s 2024-03-29T04:08:47.209 INFO:teuthology.orchestra.run.smithi097.stderr: 58900K .......... .......... .......... .......... .......... 34% 25.1M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 58950K .......... .......... .......... .......... .......... 35% 23.8M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 59000K .......... .......... .......... .......... .......... 35% 25.6M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 59050K .......... .......... .......... .......... .......... 35% 64.9M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 59100K .......... .......... .......... .......... .......... 35% 25.5M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 59150K .......... .......... .......... .......... .......... 35% 27.8M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 59200K .......... .......... .......... .......... .......... 35% 25.2M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 59250K .......... .......... .......... .......... .......... 35% 23.8M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 59300K .......... .......... .......... .......... .......... 35% 69.1M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 59350K .......... .......... .......... .......... .......... 35% 23.2M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 59400K .......... .......... .......... .......... .......... 35% 30.0M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 59450K .......... .......... .......... .......... .......... 35% 24.5M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 59500K .......... .......... .......... .......... .......... 35% 23.6M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 59550K .......... .......... .......... .......... .......... 35% 57.7M 4s 2024-03-29T04:08:47.210 INFO:teuthology.orchestra.run.smithi097.stderr: 59600K .......... .......... .......... .......... .......... 35% 24.7M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 59650K .......... .......... .......... .......... .......... 35% 31.5M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 59700K .......... .......... .......... .......... .......... 35% 24.4M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 59750K .......... .......... .......... .......... .......... 35% 23.1M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 59800K .......... .......... .......... .......... .......... 35% 53.6M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 59850K .......... .......... .......... .......... .......... 35% 23.5M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 59900K .......... .......... .......... .......... .......... 35% 33.3M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 59950K .......... .......... .......... .......... .......... 35% 19.8M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 60000K .......... .......... .......... .......... .......... 35% 33.5M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 60050K .......... .......... .......... .......... .......... 35% 49.1M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 60100K .......... .......... .......... .......... .......... 35% 27.9M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 60150K .......... .......... .......... .......... .......... 35% 23.8M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 60200K .......... .......... .......... .......... .......... 35% 37.7M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 60250K .......... .......... .......... .......... .......... 35% 20.6M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 60300K .......... .......... .......... .......... .......... 35% 41.3M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 60350K .......... .......... .......... .......... .......... 35% 28.4M 4s 2024-03-29T04:08:47.211 INFO:teuthology.orchestra.run.smithi097.stderr: 60400K .......... .......... .......... .......... .......... 35% 30.3M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 60450K .......... .......... .......... .......... .......... 35% 27.5M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 60500K .......... .......... .......... .......... .......... 35% 24.5M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 60550K .......... .......... .......... .......... .......... 35% 53.4M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 60600K .......... .......... .......... .......... .......... 35% 23.0M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 60650K .......... .......... .......... .......... .......... 36% 36.6M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 60700K .......... .......... .......... .......... .......... 36% 24.3M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 60750K .......... .......... .......... .......... .......... 36% 24.4M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 60800K .......... .......... .......... .......... .......... 36% 54.5M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 60850K .......... .......... .......... .......... .......... 36% 29.3M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 60900K .......... .......... .......... .......... .......... 36% 18.0M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 60950K .......... .......... .......... .......... .......... 36% 42.1M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 61000K .......... .......... .......... .......... .......... 36% 24.7M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 61050K .......... .......... .......... .......... .......... 36% 56.5M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 61100K .......... .......... .......... .......... .......... 36% 26.1M 4s 2024-03-29T04:08:47.212 INFO:teuthology.orchestra.run.smithi097.stderr: 61150K .......... .......... .......... .......... .......... 36% 29.6M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61200K .......... .......... .......... .......... .......... 36% 24.4M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61250K .......... .......... .......... .......... .......... 36% 11.1M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61300K .......... .......... .......... .......... .......... 36% 60.2M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61350K .......... .......... .......... .......... .......... 36% 48.6M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61400K .......... .......... .......... .......... .......... 36% 43.7M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61450K .......... .......... .......... .......... .......... 36% 47.5M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61500K .......... .......... .......... .......... .......... 36% 24.7M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61550K .......... .......... .......... .......... .......... 36% 53.7M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61600K .......... .......... .......... .......... .......... 36% 24.5M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61650K .......... .......... .......... .......... .......... 36% 31.7M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61700K .......... .......... .......... .......... .......... 36% 24.3M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61750K .......... .......... .......... .......... .......... 36% 24.7M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61800K .......... .......... .......... .......... .......... 36% 57.0M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61850K .......... .......... .......... .......... .......... 36% 22.6M 4s 2024-03-29T04:08:47.213 INFO:teuthology.orchestra.run.smithi097.stderr: 61900K .......... .......... .......... .......... .......... 36% 31.6M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 61950K .......... .......... .......... .......... .......... 36% 26.4M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 62000K .......... .......... .......... .......... .......... 36% 25.2M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 62050K .......... .......... .......... .......... .......... 36% 45.7M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 62100K .......... .......... .......... .......... .......... 36% 24.3M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 62150K .......... .......... .......... .......... .......... 36% 19.9M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 62200K .......... .......... .......... .......... .......... 36% 59.5M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 62250K .......... .......... .......... .......... .......... 36% 23.9M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 62300K .......... .......... .......... .......... .......... 36% 33.6M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 62350K .......... .......... .......... .......... .......... 37% 33.2M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 62400K .......... .......... .......... .......... .......... 37% 33.3M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 62450K .......... .......... .......... .......... .......... 37% 24.6M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 62500K .......... .......... .......... .......... .......... 37% 24.5M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 62550K .......... .......... .......... .......... .......... 37% 47.5M 4s 2024-03-29T04:08:47.214 INFO:teuthology.orchestra.run.smithi097.stderr: 62600K .......... .......... .......... .......... .......... 37% 24.6M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 62650K .......... .......... .......... .......... .......... 37% 18.7M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 62700K .......... .......... .......... .......... .......... 37% 24.1M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 62750K .......... .......... .......... .......... .......... 37% 32.4M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 62800K .......... .......... .......... .......... .......... 37% 95.2M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 62850K .......... .......... .......... .......... .......... 37% 27.9M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 62900K .......... .......... .......... .......... .......... 37% 24.6M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 62950K .......... .......... .......... .......... .......... 37% 32.7M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 63000K .......... .......... .......... .......... .......... 37% 24.2M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 63050K .......... .......... .......... .......... .......... 37% 58.0M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 63100K .......... .......... .......... .......... .......... 37% 24.9M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 63150K .......... .......... .......... .......... .......... 37% 28.8M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 63200K .......... .......... .......... .......... .......... 37% 24.7M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 63250K .......... .......... .......... .......... .......... 37% 29.1M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 63300K .......... .......... .......... .......... .......... 37% 45.2M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 63350K .......... .......... .......... .......... .......... 37% 24.3M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 63400K .......... .......... .......... .......... .......... 37% 24.5M 4s 2024-03-29T04:08:47.215 INFO:teuthology.orchestra.run.smithi097.stderr: 63450K .......... .......... .......... .......... .......... 37% 29.1M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 63500K .......... .......... .......... .......... .......... 37% 56.0M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 63550K .......... .......... .......... .......... .......... 37% 28.1M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 63600K .......... .......... .......... .......... .......... 37% 24.8M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 63650K .......... .......... .......... .......... .......... 37% 26.1M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 63700K .......... .......... .......... .......... .......... 37% 20.2M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 63750K .......... .......... .......... .......... .......... 37% 103M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 63800K .......... .......... .......... .......... .......... 37% 30.4M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 63850K .......... .......... .......... .......... .......... 37% 23.7M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 63900K .......... .......... .......... .......... .......... 37% 20.5M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 63950K .......... .......... .......... .......... .......... 37% 30.9M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 64000K .......... .......... .......... .......... .......... 37% 34.2M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 64050K .......... .......... .......... .......... .......... 38% 24.7M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 64100K .......... .......... .......... .......... .......... 38% 50.5M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 64150K .......... .......... .......... .......... .......... 38% 23.8M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 64200K .......... .......... .......... .......... .......... 38% 26.2M 4s 2024-03-29T04:08:47.216 INFO:teuthology.orchestra.run.smithi097.stderr: 64250K .......... .......... .......... .......... .......... 38% 31.9M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64300K .......... .......... .......... .......... .......... 38% 32.4M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64350K .......... .......... .......... .......... .......... 38% 33.6M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64400K .......... .......... .......... .......... .......... 38% 25.1M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64450K .......... .......... .......... .......... .......... 38% 25.3M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64500K .......... .......... .......... .......... .......... 38% 40.2M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64550K .......... .......... .......... .......... .......... 38% 27.5M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64600K .......... .......... .......... .......... .......... 38% 32.0M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64650K .......... .......... .......... .......... .......... 38% 28.2M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64700K .......... .......... .......... .......... .......... 38% 28.8M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64750K .......... .......... .......... .......... .......... 38% 21.8M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64800K .......... .......... .......... .......... .......... 38% 36.1M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64850K .......... .......... .......... .......... .......... 38% 35.1M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64900K .......... .......... .......... .......... .......... 38% 22.7M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 64950K .......... .......... .......... .......... .......... 38% 40.7M 4s 2024-03-29T04:08:47.217 INFO:teuthology.orchestra.run.smithi097.stderr: 65000K .......... .......... .......... .......... .......... 38% 23.4M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65050K .......... .......... .......... .......... .......... 38% 20.7M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65100K .......... .......... .......... .......... .......... 38% 71.1M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65150K .......... .......... .......... .......... .......... 38% 14.5M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65200K .......... .......... .......... .......... .......... 38% 60.3M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65250K .......... .......... .......... .......... .......... 38% 46.4M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65300K .......... .......... .......... .......... .......... 38% 26.4M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65350K .......... .......... .......... .......... .......... 38% 33.1M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65400K .......... .......... .......... .......... .......... 38% 37.6M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65450K .......... .......... .......... .......... .......... 38% 24.9M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65500K .......... .......... .......... .......... .......... 38% 25.0M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65550K .......... .......... .......... .......... .......... 38% 25.9M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65600K .......... .......... .......... .......... .......... 38% 45.4M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65650K .......... .......... .......... .......... .......... 38% 31.1M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65700K .......... .......... .......... .......... .......... 39% 25.9M 4s 2024-03-29T04:08:47.218 INFO:teuthology.orchestra.run.smithi097.stderr: 65750K .......... .......... .......... .......... .......... 39% 25.2M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 65800K .......... .......... .......... .......... .......... 39% 23.6M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 65850K .......... .......... .......... .......... .......... 39% 53.2M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 65900K .......... .......... .......... .......... .......... 39% 26.2M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 65950K .......... .......... .......... .......... .......... 39% 14.1M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 66000K .......... .......... .......... .......... .......... 39% 29.9M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 66050K .......... .......... .......... .......... .......... 39% 19.3M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 66100K .......... .......... .......... .......... .......... 39% 56.7M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 66150K .......... .......... .......... .......... .......... 39% 31.4M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 66200K .......... .......... .......... .......... .......... 39% 16.1M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 66250K .......... .......... .......... .......... .......... 39% 16.4M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 66300K .......... .......... .......... .......... .......... 39% 22.9M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 66350K .......... .......... .......... .......... .......... 39% 40.2M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 66400K .......... .......... .......... .......... .......... 39% 19.6M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 66450K .......... .......... .......... .......... .......... 39% 27.1M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 66500K .......... .......... .......... .......... .......... 39% 23.2M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 66550K .......... .......... .......... .......... .......... 39% 19.4M 4s 2024-03-29T04:08:47.219 INFO:teuthology.orchestra.run.smithi097.stderr: 66600K .......... .......... .......... .......... .......... 39% 45.9M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 66650K .......... .......... .......... .......... .......... 39% 32.6M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 66700K .......... .......... .......... .......... .......... 39% 43.0M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 66750K .......... .......... .......... .......... .......... 39% 32.2M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 66800K .......... .......... .......... .......... .......... 39% 26.6M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 66850K .......... .......... .......... .......... .......... 39% 34.8M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 66900K .......... .......... .......... .......... .......... 39% 17.9M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 66950K .......... .......... .......... .......... .......... 39% 33.3M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 67000K .......... .......... .......... .......... .......... 39% 18.5M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 67050K .......... .......... .......... .......... .......... 39% 23.8M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 67100K .......... .......... .......... .......... .......... 39% 76.3M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 67150K .......... .......... .......... .......... .......... 39% 90.2M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 67200K .......... .......... .......... .......... .......... 39% 28.6M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 67250K .......... .......... .......... .......... .......... 39% 12.3M 4s 2024-03-29T04:08:47.220 INFO:teuthology.orchestra.run.smithi097.stderr: 67300K .......... .......... .......... .......... .......... 39% 56.7M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 67350K .......... .......... .......... .......... .......... 39% 43.5M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 67400K .......... .......... .......... .......... .......... 40% 48.2M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 67450K .......... .......... .......... .......... .......... 40% 73.4M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 67500K .......... .......... .......... .......... .......... 40% 46.2M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 67550K .......... .......... .......... .......... .......... 40% 106M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 67600K .......... .......... .......... .......... .......... 40% 96.5M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 67650K .......... .......... .......... .......... .......... 40% 53.3M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 67700K .......... .......... .......... .......... .......... 40% 42.9M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 67750K .......... .......... .......... .......... .......... 40% 23.9M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 67800K .......... .......... .......... .......... .......... 40% 25.0M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 67850K .......... .......... .......... .......... .......... 40% 54.0M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 67900K .......... .......... .......... .......... .......... 40% 29.8M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 67950K .......... .......... .......... .......... .......... 40% 20.2M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 68000K .......... .......... .......... .......... .......... 40% 38.1M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 68050K .......... .......... .......... .......... .......... 40% 21.2M 4s 2024-03-29T04:08:47.221 INFO:teuthology.orchestra.run.smithi097.stderr: 68100K .......... .......... .......... .......... .......... 40% 44.8M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68150K .......... .......... .......... .......... .......... 40% 27.2M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68200K .......... .......... .......... .......... .......... 40% 24.4M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68250K .......... .......... .......... .......... .......... 40% 33.1M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68300K .......... .......... .......... .......... .......... 40% 24.4M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68350K .......... .......... .......... .......... .......... 40% 51.1M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68400K .......... .......... .......... .......... .......... 40% 24.8M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68450K .......... .......... .......... .......... .......... 40% 23.5M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68500K .......... .......... .......... .......... .......... 40% 37.6M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68550K .......... .......... .......... .......... .......... 40% 13.5M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68600K .......... .......... .......... .......... .......... 40% 43.2M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68650K .......... .......... .......... .......... .......... 40% 11.9M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68700K .......... .......... .......... .......... .......... 40% 103M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68750K .......... .......... .......... .......... .......... 40% 75.1M 4s 2024-03-29T04:08:47.222 INFO:teuthology.orchestra.run.smithi097.stderr: 68800K .......... .......... .......... .......... .......... 40% 100M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 68850K .......... .......... .......... .......... .......... 40% 43.7M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 68900K .......... .......... .......... .......... .......... 40% 24.0M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 68950K .......... .......... .......... .......... .......... 40% 24.5M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 69000K .......... .......... .......... .......... .......... 40% 29.3M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 69050K .......... .......... .......... .......... .......... 40% 30.9M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 69100K .......... .......... .......... .......... .......... 41% 40.7M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 69150K .......... .......... .......... .......... .......... 41% 25.3M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 69200K .......... .......... .......... .......... .......... 41% 38.7M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 69250K .......... .......... .......... .......... .......... 41% 24.2M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 69300K .......... .......... .......... .......... .......... 41% 24.2M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 69350K .......... .......... .......... .......... .......... 41% 24.8M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 69400K .......... .......... .......... .......... .......... 41% 40.5M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 69450K .......... .......... .......... .......... .......... 41% 24.5M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 69500K .......... .......... .......... .......... .......... 41% 40.0M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 69550K .......... .......... .......... .......... .......... 41% 24.0M 4s 2024-03-29T04:08:47.223 INFO:teuthology.orchestra.run.smithi097.stderr: 69600K .......... .......... .......... .......... .......... 41% 17.5M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 69650K .......... .......... .......... .......... .......... 41% 64.6M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 69700K .......... .......... .......... .......... .......... 41% 32.3M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 69750K .......... .......... .......... .......... .......... 41% 30.3M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 69800K .......... .......... .......... .......... .......... 41% 26.4M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 69850K .......... .......... .......... .......... .......... 41% 49.2M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 69900K .......... .......... .......... .......... .......... 41% 26.4M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 69950K .......... .......... .......... .......... .......... 41% 28.1M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 70000K .......... .......... .......... .......... .......... 41% 23.2M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 70050K .......... .......... .......... .......... .......... 41% 28.7M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 70100K .......... .......... .......... .......... .......... 41% 28.7M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 70150K .......... .......... .......... .......... .......... 41% 38.7M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 70200K .......... .......... .......... .......... .......... 41% 24.1M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 70250K .......... .......... .......... .......... .......... 41% 29.2M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 70300K .......... .......... .......... .......... .......... 41% 23.0M 4s 2024-03-29T04:08:47.224 INFO:teuthology.orchestra.run.smithi097.stderr: 70350K .......... .......... .......... .......... .......... 41% 68.8M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 70400K .......... .......... .......... .......... .......... 41% 29.1M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 70450K .......... .......... .......... .......... .......... 41% 24.3M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 70500K .......... .......... .......... .......... .......... 41% 22.7M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 70550K .......... .......... .......... .......... .......... 41% 27.4M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 70600K .......... .......... .......... .......... .......... 41% 30.6M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 70650K .......... .......... .......... .......... .......... 41% 56.0M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 70700K .......... .......... .......... .......... .......... 41% 22.9M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 70750K .......... .......... .......... .......... .......... 42% 34.2M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 70800K .......... .......... .......... .......... .......... 42% 25.2M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 70850K .......... .......... .......... .......... .......... 42% 30.3M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 70900K .......... .......... .......... .......... .......... 42% 38.6M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 70950K .......... .......... .......... .......... .......... 42% 24.7M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 71000K .......... .......... .......... .......... .......... 42% 22.6M 4s 2024-03-29T04:08:47.225 INFO:teuthology.orchestra.run.smithi097.stderr: 71050K .......... .......... .......... .......... .......... 42% 65.6M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71100K .......... .......... .......... .......... .......... 42% 23.3M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71150K .......... .......... .......... .......... .......... 42% 32.2M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71200K .......... .......... .......... .......... .......... 42% 25.2M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71250K .......... .......... .......... .......... .......... 42% 27.4M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71300K .......... .......... .......... .......... .......... 42% 36.9M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71350K .......... .......... .......... .......... .......... 42% 23.1M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71400K .......... .......... .......... .......... .......... 42% 36.9M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71450K .......... .......... .......... .......... .......... 42% 20.7M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71500K .......... .......... .......... .......... .......... 42% 49.2M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71550K .......... .......... .......... .......... .......... 42% 28.2M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71600K .......... .......... .......... .......... .......... 42% 27.2M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71650K .......... .......... .......... .......... .......... 42% 36.2M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71700K .......... .......... .......... .......... .......... 42% 23.9M 4s 2024-03-29T04:08:47.226 INFO:teuthology.orchestra.run.smithi097.stderr: 71750K .......... .......... .......... .......... .......... 42% 34.9M 4s 2024-03-29T04:08:47.227 INFO:teuthology.orchestra.run.smithi097.stderr: 71800K .......... .......... .......... .......... .......... 42% 25.8M 4s 2024-03-29T04:08:47.227 INFO:teuthology.orchestra.run.smithi097.stderr: 71850K .......... .......... .......... .......... .......... 42% 25.0M 4s 2024-03-29T04:08:47.227 INFO:teuthology.orchestra.run.smithi097.stderr: 71900K .......... .......... .......... .......... .......... 42% 41.9M 4s 2024-03-29T04:08:47.227 INFO:teuthology.orchestra.run.smithi097.stderr: 71950K .......... .......... .......... .......... .......... 42% 25.0M 4s 2024-03-29T04:08:47.227 INFO:teuthology.orchestra.run.smithi097.stderr: 72000K .......... .......... .......... .......... .......... 42% 34.6M 4s 2024-03-29T04:08:47.227 INFO:teuthology.orchestra.run.smithi097.stderr: 72050K .......... .......... .......... .......... .......... 42% 25.0M 4s 2024-03-29T04:08:47.227 INFO:teuthology.orchestra.run.smithi097.stderr: 72100K .......... .......... .......... .......... .......... 42% 13.4M 4s 2024-03-29T04:08:47.227 INFO:teuthology.orchestra.run.smithi097.stderr: 72150K .......... .......... .......... .......... .......... 42% 83.3M 4s 2024-03-29T04:08:47.227 INFO:teuthology.orchestra.run.smithi097.stderr: 72200K .......... .......... .......... .......... .......... 42% 69.7M 4s 2024-03-29T04:08:47.227 INFO:teuthology.orchestra.run.smithi097.stderr: 72250K .......... .......... .......... .......... .......... 42% 33.9M 4s 2024-03-29T04:08:47.227 INFO:teuthology.orchestra.run.smithi097.stderr: 72300K .......... .......... .......... .......... .......... 42% 24.1M 4s 2024-03-29T04:08:47.227 INFO:teuthology.orchestra.run.smithi097.stderr: 72350K .......... .......... .......... .......... .......... 42% 24.9M 4s 2024-03-29T04:08:47.227 INFO:teuthology.orchestra.run.smithi097.stderr: 72400K .......... .......... .......... .......... .......... 42% 52.0M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 72450K .......... .......... .......... .......... .......... 43% 29.5M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 72500K .......... .......... .......... .......... .......... 43% 26.7M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 72550K .......... .......... .......... .......... .......... 43% 24.8M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 72600K .......... .......... .......... .......... .......... 43% 26.5M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 72650K .......... .......... .......... .......... .......... 43% 40.2M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 72700K .......... .......... .......... .......... .......... 43% 25.3M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 72750K .......... .......... .......... .......... .......... 43% 36.5M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 72800K .......... .......... .......... .......... .......... 43% 26.5M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 72850K .......... .......... .......... .......... .......... 43% 23.7M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 72900K .......... .......... .......... .......... .......... 43% 40.5M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 72950K .......... .......... .......... .......... .......... 43% 30.4M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 73000K .......... .......... .......... .......... .......... 43% 28.4M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 73050K .......... .......... .......... .......... .......... 43% 25.1M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 73100K .......... .......... .......... .......... .......... 43% 24.5M 4s 2024-03-29T04:08:47.228 INFO:teuthology.orchestra.run.smithi097.stderr: 73150K .......... .......... .......... .......... .......... 43% 55.5M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73200K .......... .......... .......... .......... .......... 43% 23.4M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73250K .......... .......... .......... .......... .......... 43% 34.0M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73300K .......... .......... .......... .......... .......... 43% 24.6M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73350K .......... .......... .......... .......... .......... 43% 24.4M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73400K .......... .......... .......... .......... .......... 43% 57.6M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73450K .......... .......... .......... .......... .......... 43% 22.9M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73500K .......... .......... .......... .......... .......... 43% 35.4M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73550K .......... .......... .......... .......... .......... 43% 24.5M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73600K .......... .......... .......... .......... .......... 43% 28.5M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73650K .......... .......... .......... .......... .......... 43% 27.7M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73700K .......... .......... .......... .......... .......... 43% 33.6M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73750K .......... .......... .......... .......... .......... 43% 30.4M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73800K .......... .......... .......... .......... .......... 43% 26.0M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73850K .......... .......... .......... .......... .......... 43% 25.1M 4s 2024-03-29T04:08:47.229 INFO:teuthology.orchestra.run.smithi097.stderr: 73900K .......... .......... .......... .......... .......... 43% 41.4M 4s 2024-03-29T04:08:47.230 INFO:teuthology.orchestra.run.smithi097.stderr: 73950K .......... .......... .......... .......... .......... 43% 27.5M 4s 2024-03-29T04:08:47.230 INFO:teuthology.orchestra.run.smithi097.stderr: 74000K .......... .......... .......... .......... .......... 43% 30.1M 4s 2024-03-29T04:08:47.230 INFO:teuthology.orchestra.run.smithi097.stderr: 74050K .......... .......... .......... .......... .......... 43% 27.1M 4s 2024-03-29T04:08:47.230 INFO:teuthology.orchestra.run.smithi097.stderr: 74100K .......... .......... .......... .......... .......... 43% 32.7M 4s 2024-03-29T04:08:47.230 INFO:teuthology.orchestra.run.smithi097.stderr: 74150K .......... .......... .......... .......... .......... 44% 31.4M 4s 2024-03-29T04:08:47.230 INFO:teuthology.orchestra.run.smithi097.stderr: 74200K .......... .......... .......... .......... .......... 44% 23.7M 4s 2024-03-29T04:08:47.230 INFO:teuthology.orchestra.run.smithi097.stderr: 74250K .......... .......... .......... .......... .......... 44% 33.5M 4s 2024-03-29T04:08:47.230 INFO:teuthology.orchestra.run.smithi097.stderr: 74300K .......... .......... .......... .......... .......... 44% 25.7M 4s 2024-03-29T04:08:47.230 INFO:teuthology.orchestra.run.smithi097.stderr: 74350K .......... .......... .......... .......... .......... 44% 24.1M 4s 2024-03-29T04:08:47.230 INFO:teuthology.orchestra.run.smithi097.stderr: 74400K .......... .......... .......... .......... .......... 44% 52.2M 4s 2024-03-29T04:08:47.230 INFO:teuthology.orchestra.run.smithi097.stderr: 74450K .......... .......... .......... .......... .......... 44% 23.9M 4s 2024-03-29T04:08:47.230 INFO:teuthology.orchestra.run.smithi097.stderr: 74500K .......... .......... .......... .......... .......... 44% 24.8M 4s 2024-03-29T04:08:47.230 INFO:teuthology.orchestra.run.smithi097.stderr: 74550K .......... .......... .......... .......... .......... 44% 23.5M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 74600K .......... .......... .......... .......... .......... 44% 18.1M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 74650K .......... .......... .......... .......... .......... 44% 22.4M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 74700K .......... .......... .......... .......... .......... 44% 41.5M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 74750K .......... .......... .......... .......... .......... 44% 73.2M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 74800K .......... .......... .......... .......... .......... 44% 46.3M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 74850K .......... .......... .......... .......... .......... 44% 29.7M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 74900K .......... .......... .......... .......... .......... 44% 20.7M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 74950K .......... .......... .......... .......... .......... 44% 51.6M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 75000K .......... .......... .......... .......... .......... 44% 26.7M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 75050K .......... .......... .......... .......... .......... 44% 29.6M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 75100K .......... .......... .......... .......... .......... 44% 25.1M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 75150K .......... .......... .......... .......... .......... 44% 24.7M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 75200K .......... .......... .......... .......... .......... 44% 59.7M 4s 2024-03-29T04:08:47.231 INFO:teuthology.orchestra.run.smithi097.stderr: 75250K .......... .......... .......... .......... .......... 44% 23.6M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75300K .......... .......... .......... .......... .......... 44% 29.2M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75350K .......... .......... .......... .......... .......... 44% 24.9M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75400K .......... .......... .......... .......... .......... 44% 24.5M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75450K .......... .......... .......... .......... .......... 44% 54.5M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75500K .......... .......... .......... .......... .......... 44% 24.5M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75550K .......... .......... .......... .......... .......... 44% 32.5M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75600K .......... .......... .......... .......... .......... 44% 24.5M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75650K .......... .......... .......... .......... .......... 44% 29.9M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75700K .......... .......... .......... .......... .......... 44% 34.8M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75750K .......... .......... .......... .......... .......... 44% 27.0M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75800K .......... .......... .......... .......... .......... 44% 31.8M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75850K .......... .......... .......... .......... .......... 45% 24.7M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75900K .......... .......... .......... .......... .......... 45% 25.0M 4s 2024-03-29T04:08:47.232 INFO:teuthology.orchestra.run.smithi097.stderr: 75950K .......... .......... .......... .......... .......... 45% 41.9M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76000K .......... .......... .......... .......... .......... 45% 24.3M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76050K .......... .......... .......... .......... .......... 45% 37.9M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76100K .......... .......... .......... .......... .......... 45% 22.7M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76150K .......... .......... .......... .......... .......... 45% 38.2M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76200K .......... .......... .......... .......... .......... 45% 34.2M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76250K .......... .......... .......... .......... .......... 45% 23.2M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76300K .......... .......... .......... .......... .......... 45% 36.0M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76350K .......... .......... .......... .......... .......... 45% 24.1M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76400K .......... .......... .......... .......... .......... 45% 25.0M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76450K .......... .......... .......... .......... .......... 45% 46.9M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76500K .......... .......... .......... .......... .......... 45% 20.8M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76550K .......... .......... .......... .......... .......... 45% 27.3M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76600K .......... .......... .......... .......... .......... 45% 20.9M 4s 2024-03-29T04:08:47.233 INFO:teuthology.orchestra.run.smithi097.stderr: 76650K .......... .......... .......... .......... .......... 45% 57.1M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 76700K .......... .......... .......... .......... .......... 45% 61.8M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 76750K .......... .......... .......... .......... .......... 45% 21.8M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 76800K .......... .......... .......... .......... .......... 45% 35.1M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 76850K .......... .......... .......... .......... .......... 45% 22.4M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 76900K .......... .......... .......... .......... .......... 45% 24.7M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 76950K .......... .......... .......... .......... .......... 45% 44.4M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 77000K .......... .......... .......... .......... .......... 45% 31.4M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 77050K .......... .......... .......... .......... .......... 45% 19.7M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 77100K .......... .......... .......... .......... .......... 45% 33.3M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 77150K .......... .......... .......... .......... .......... 45% 57.3M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 77200K .......... .......... .......... .......... .......... 45% 14.1M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 77250K .......... .......... .......... .......... .......... 45% 198M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 77300K .......... .......... .......... .......... .......... 45% 25.1M 4s 2024-03-29T04:08:47.234 INFO:teuthology.orchestra.run.smithi097.stderr: 77350K .......... .......... .......... .......... .......... 45% 25.1M 4s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 77400K .......... .......... .......... .......... .......... 45% 45.8M 4s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 77450K .......... .......... .......... .......... .......... 45% 32.2M 4s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 77500K .......... .......... .......... .......... .......... 46% 25.9M 4s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 77550K .......... .......... .......... .......... .......... 46% 24.9M 4s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 77600K .......... .......... .......... .......... .......... 46% 24.5M 4s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 77650K .......... .......... .......... .......... .......... 46% 48.8M 4s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 77700K .......... .......... .......... .......... .......... 46% 31.3M 4s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 77750K .......... .......... .......... .......... .......... 46% 23.5M 4s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 77800K .......... .......... .......... .......... .......... 46% 28.0M 4s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 77850K .......... .......... .......... .......... .......... 46% 23.0M 4s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 77900K .......... .......... .......... .......... .......... 46% 55.3M 4s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 77950K .......... .......... .......... .......... .......... 46% 30.4M 3s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 78000K .......... .......... .......... .......... .......... 46% 25.8M 3s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 78050K .......... .......... .......... .......... .......... 46% 23.2M 3s 2024-03-29T04:08:47.235 INFO:teuthology.orchestra.run.smithi097.stderr: 78100K .......... .......... .......... .......... .......... 46% 35.7M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78150K .......... .......... .......... .......... .......... 46% 24.7M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78200K .......... .......... .......... .......... .......... 46% 42.6M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78250K .......... .......... .......... .......... .......... 46% 23.3M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78300K .......... .......... .......... .......... .......... 46% 28.5M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78350K .......... .......... .......... .......... .......... 46% 22.7M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78400K .......... .......... .......... .......... .......... 46% 46.5M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78450K .......... .......... .......... .......... .......... 46% 34.9M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78500K .......... .......... .......... .......... .......... 46% 25.1M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78550K .......... .......... .......... .......... .......... 46% 24.0M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78600K .......... .......... .......... .......... .......... 46% 46.9M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78650K .......... .......... .......... .......... .......... 46% 24.1M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78700K .......... .......... .......... .......... .......... 46% 35.1M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78750K .......... .......... .......... .......... .......... 46% 23.2M 3s 2024-03-29T04:08:47.236 INFO:teuthology.orchestra.run.smithi097.stderr: 78800K .......... .......... .......... .......... .......... 46% 29.6M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 78850K .......... .......... .......... .......... .......... 46% 35.7M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 78900K .......... .......... .......... .......... .......... 46% 24.0M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 78950K .......... .......... .......... .......... .......... 46% 36.4M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 79000K .......... .......... .......... .......... .......... 46% 25.1M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 79050K .......... .......... .......... .......... .......... 46% 24.3M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 79100K .......... .......... .......... .......... .......... 46% 42.2M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 79150K .......... .......... .......... .......... .......... 46% 22.1M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 79200K .......... .......... .......... .......... .......... 47% 41.0M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 79250K .......... .......... .......... .......... .......... 47% 24.1M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 79300K .......... .......... .......... .......... .......... 47% 24.6M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 79350K .......... .......... .......... .......... .......... 47% 35.6M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 79400K .......... .......... .......... .......... .......... 47% 24.4M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 79450K .......... .......... .......... .......... .......... 47% 51.5M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 79500K .......... .......... .......... .......... .......... 47% 24.5M 3s 2024-03-29T04:08:47.237 INFO:teuthology.orchestra.run.smithi097.stderr: 79550K .......... .......... .......... .......... .......... 47% 28.9M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 79600K .......... .......... .......... .......... .......... 47% 26.3M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 79650K .......... .......... .......... .......... .......... 47% 24.5M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 79700K .......... .......... .......... .......... .......... 47% 51.9M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 79750K .......... .......... .......... .......... .......... 47% 24.3M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 79800K .......... .......... .......... .......... .......... 47% 21.4M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 79850K .......... .......... .......... .......... .......... 47% 47.1M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 79900K .......... .......... .......... .......... .......... 47% 22.9M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 79950K .......... .......... .......... .......... .......... 47% 24.9M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 80000K .......... .......... .......... .......... .......... 47% 52.4M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 80050K .......... .......... .......... .......... .......... 47% 29.6M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 80100K .......... .......... .......... .......... .......... 47% 26.4M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 80150K .......... .......... .......... .......... .......... 47% 25.5M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 80200K .......... .......... .......... .......... .......... 47% 45.0M 3s 2024-03-29T04:08:47.238 INFO:teuthology.orchestra.run.smithi097.stderr: 80250K .......... .......... .......... .......... .......... 47% 22.8M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80300K .......... .......... .......... .......... .......... 47% 25.1M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80350K .......... .......... .......... .......... .......... 47% 40.0M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80400K .......... .......... .......... .......... .......... 47% 20.1M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80450K .......... .......... .......... .......... .......... 47% 30.4M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80500K .......... .......... .......... .......... .......... 47% 50.9M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80550K .......... .......... .......... .......... .......... 47% 25.2M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80600K .......... .......... .......... .......... .......... 47% 33.6M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80650K .......... .......... .......... .......... .......... 47% 24.5M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80700K .......... .......... .......... .......... .......... 47% 23.8M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80750K .......... .......... .......... .......... .......... 47% 50.1M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80800K .......... .......... .......... .......... .......... 47% 34.7M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80850K .......... .......... .......... .......... .......... 47% 24.8M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80900K .......... .......... .......... .......... .......... 48% 24.3M 3s 2024-03-29T04:08:47.239 INFO:teuthology.orchestra.run.smithi097.stderr: 80950K .......... .......... .......... .......... .......... 48% 41.9M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81000K .......... .......... .......... .......... .......... 48% 26.6M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81050K .......... .......... .......... .......... .......... 48% 24.0M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81100K .......... .......... .......... .......... .......... 48% 32.8M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81150K .......... .......... .......... .......... .......... 48% 25.9M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81200K .......... .......... .......... .......... .......... 48% 23.8M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81250K .......... .......... .......... .......... .......... 48% 33.2M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81300K .......... .......... .......... .......... .......... 48% 49.5M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81350K .......... .......... .......... .......... .......... 48% 25.6M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81400K .......... .......... .......... .......... .......... 48% 23.9M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81450K .......... .......... .......... .......... .......... 48% 24.8M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81500K .......... .......... .......... .......... .......... 48% 52.5M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81550K .......... .......... .......... .......... .......... 48% 31.2M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81600K .......... .......... .......... .......... .......... 48% 25.2M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81650K .......... .......... .......... .......... .......... 48% 24.7M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81700K .......... .......... .......... .......... .......... 48% 24.0M 3s 2024-03-29T04:08:47.240 INFO:teuthology.orchestra.run.smithi097.stderr: 81750K .......... .......... .......... .......... .......... 48% 56.2M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 81800K .......... .......... .......... .......... .......... 48% 28.5M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 81850K .......... .......... .......... .......... .......... 48% 28.2M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 81900K .......... .......... .......... .......... .......... 48% 21.5M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 81950K .......... .......... .......... .......... .......... 48% 29.1M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 82000K .......... .......... .......... .......... .......... 48% 51.9M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 82050K .......... .......... .......... .......... .......... 48% 30.0M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 82100K .......... .......... .......... .......... .......... 48% 26.2M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 82150K .......... .......... .......... .......... .......... 48% 24.2M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 82200K .......... .......... .......... .......... .......... 48% 24.2M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 82250K .......... .......... .......... .......... .......... 48% 62.1M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 82300K .......... .......... .......... .......... .......... 48% 24.1M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 82350K .......... .......... .......... .......... .......... 48% 32.8M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 82400K .......... .......... .......... .......... .......... 48% 27.3M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 82450K .......... .......... .......... .......... .......... 48% 21.8M 3s 2024-03-29T04:08:47.241 INFO:teuthology.orchestra.run.smithi097.stderr: 82500K .......... .......... .......... .......... .......... 48% 57.8M 3s 2024-03-29T04:08:47.242 INFO:teuthology.orchestra.run.smithi097.stderr: 82550K .......... .......... .......... .......... .......... 49% 24.8M 3s 2024-03-29T04:08:47.242 INFO:teuthology.orchestra.run.smithi097.stderr: 82600K .......... .......... .......... .......... .......... 49% 31.1M 3s 2024-03-29T04:08:47.242 INFO:teuthology.orchestra.run.smithi097.stderr: 82650K .......... .......... .......... .......... .......... 49% 25.2M 3s 2024-03-29T04:08:47.242 INFO:teuthology.orchestra.run.smithi097.stderr: 82700K .......... .......... .......... .......... .......... 49% 24.0M 3s 2024-03-29T04:08:47.242 INFO:teuthology.orchestra.run.smithi097.stderr: 82750K .......... .......... .......... .......... .......... 49% 56.1M 3s 2024-03-29T04:08:47.242 INFO:teuthology.orchestra.run.smithi097.stderr: 82800K .......... .......... .......... .......... .......... 49% 23.6M 3s 2024-03-29T04:08:47.242 INFO:teuthology.orchestra.run.smithi097.stderr: 82850K .......... .......... .......... .......... .......... 49% 33.7M 3s 2024-03-29T04:08:47.242 INFO:teuthology.orchestra.run.smithi097.stderr: 82900K .......... .......... .......... .......... .......... 49% 24.6M 3s 2024-03-29T04:08:47.242 INFO:teuthology.orchestra.run.smithi097.stderr: 82950K .......... .......... .......... .......... .......... 49% 23.9M 3s 2024-03-29T04:08:47.242 INFO:teuthology.orchestra.run.smithi097.stderr: 83000K .......... .......... .......... .......... .......... 49% 45.2M 3s 2024-03-29T04:08:47.242 INFO:teuthology.orchestra.run.smithi097.stderr: 83050K .......... .......... .......... .......... .......... 49% 18.3M 3s 2024-03-29T04:08:47.242 INFO:teuthology.orchestra.run.smithi097.stderr: 83100K .......... .......... .......... .......... .......... 49% 24.5M 3s 2024-03-29T04:08:47.242 INFO:teuthology.orchestra.run.smithi097.stderr: 83150K .......... .......... .......... .......... .......... 49% 27.3M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83200K .......... .......... .......... .......... .......... 49% 17.9M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83250K .......... .......... .......... .......... .......... 49% 29.5M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83300K .......... .......... .......... .......... .......... 49% 18.1M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83350K .......... .......... .......... .......... .......... 49% 29.8M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83400K .......... .......... .......... .......... .......... 49% 19.3M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83450K .......... .......... .......... .......... .......... 49% 19.2M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83500K .......... .......... .......... .......... .......... 49% 35.0M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83550K .......... .......... .......... .......... .......... 49% 58.4M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83600K .......... .......... .......... .......... .......... 49% 72.7M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83650K .......... .......... .......... .......... .......... 49% 99.6M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83700K .......... .......... .......... .......... .......... 49% 103M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83750K .......... .......... .......... .......... .......... 49% 46.2M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83800K .......... .......... .......... .......... .......... 49% 25.2M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83850K .......... .......... .......... .......... .......... 49% 31.5M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83900K .......... .......... .......... .......... .......... 49% 24.7M 3s 2024-03-29T04:08:47.243 INFO:teuthology.orchestra.run.smithi097.stderr: 83950K .......... .......... .......... .......... .......... 49% 24.3M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84000K .......... .......... .......... .......... .......... 49% 55.0M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84050K .......... .......... .......... .......... .......... 49% 25.8M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84100K .......... .......... .......... .......... .......... 49% 28.7M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84150K .......... .......... .......... .......... .......... 49% 24.7M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84200K .......... .......... .......... .......... .......... 49% 25.6M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84250K .......... .......... .......... .......... .......... 50% 56.5M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84300K .......... .......... .......... .......... .......... 50% 24.7M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84350K .......... .......... .......... .......... .......... 50% 30.0M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84400K .......... .......... .......... .......... .......... 50% 23.3M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84450K .......... .......... .......... .......... .......... 50% 25.7M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84500K .......... .......... .......... .......... .......... 50% 60.7M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84550K .......... .......... .......... .......... .......... 50% 24.0M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84600K .......... .......... .......... .......... .......... 50% 30.0M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84650K .......... .......... .......... .......... .......... 50% 25.1M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84700K .......... .......... .......... .......... .......... 50% 24.6M 3s 2024-03-29T04:08:47.244 INFO:teuthology.orchestra.run.smithi097.stderr: 84750K .......... .......... .......... .......... .......... 50% 71.9M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 84800K .......... .......... .......... .......... .......... 50% 25.1M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 84850K .......... .......... .......... .......... .......... 50% 22.1M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 84900K .......... .......... .......... .......... .......... 50% 24.8M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 84950K .......... .......... .......... .......... .......... 50% 35.9M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 85000K .......... .......... .......... .......... .......... 50% 38.2M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 85050K .......... .......... .......... .......... .......... 50% 1.29M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 85100K .......... .......... .......... .......... .......... 50% 112M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 85150K .......... .......... .......... .......... .......... 50% 109M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 85200K .......... .......... .......... .......... .......... 50% 106M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 85250K .......... .......... .......... .......... .......... 50% 121M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 85300K .......... .......... .......... .......... .......... 50% 96.2M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 85350K .......... .......... .......... .......... .......... 50% 105M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 85400K .......... .......... .......... .......... .......... 50% 111M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 85450K .......... .......... .......... .......... .......... 50% 27.1M 3s 2024-03-29T04:08:47.245 INFO:teuthology.orchestra.run.smithi097.stderr: 85500K .......... .......... .......... .......... .......... 50% 25.2M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 85550K .......... .......... .......... .......... .......... 50% 24.0M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 85600K .......... .......... .......... .......... .......... 50% 25.4M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 85650K .......... .......... .......... .......... .......... 50% 111M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 85700K .......... .......... .......... .......... .......... 50% 22.5M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 85750K .......... .......... .......... .......... .......... 50% 25.9M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 85800K .......... .......... .......... .......... .......... 50% 22.7M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 85850K .......... .......... .......... .......... .......... 50% 113M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 85900K .......... .......... .......... .......... .......... 50% 26.0M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 85950K .......... .......... .......... .......... .......... 51% 22.2M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 86000K .......... .......... .......... .......... .......... 51% 26.5M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 86050K .......... .......... .......... .......... .......... 51% 22.4M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 86100K .......... .......... .......... .......... .......... 51% 111M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 86150K .......... .......... .......... .......... .......... 51% 26.0M 3s 2024-03-29T04:08:47.246 INFO:teuthology.orchestra.run.smithi097.stderr: 86200K .......... .......... .......... .......... .......... 51% 22.2M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86250K .......... .......... .......... .......... .......... 51% 25.8M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86300K .......... .......... .......... .......... .......... 51% 110M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86350K .......... .......... .......... .......... .......... 51% 22.9M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86400K .......... .......... .......... .......... .......... 51% 25.4M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86450K .......... .......... .......... .......... .......... 51% 23.6M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86500K .......... .......... .......... .......... .......... 51% 25.9M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86550K .......... .......... .......... .......... .......... 51% 110M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86600K .......... .......... .......... .......... .......... 51% 22.5M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86650K .......... .......... .......... .......... .......... 51% 25.7M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86700K .......... .......... .......... .......... .......... 51% 22.6M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86750K .......... .......... .......... .......... .......... 51% 26.0M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86800K .......... .......... .......... .......... .......... 51% 120M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86850K .......... .......... .......... .......... .......... 51% 22.4M 3s 2024-03-29T04:08:47.247 INFO:teuthology.orchestra.run.smithi097.stderr: 86900K .......... .......... .......... .......... .......... 51% 24.9M 3s 2024-03-29T04:08:47.248 INFO:teuthology.orchestra.run.smithi097.stderr: 86950K .......... .......... .......... .......... .......... 51% 22.8M 3s 2024-03-29T04:08:47.248 INFO:teuthology.orchestra.run.smithi097.stderr: 87000K .......... .......... .......... .......... .......... 51% 115M 3s 2024-03-29T04:08:47.248 INFO:teuthology.orchestra.run.smithi097.stderr: 87050K .......... .......... .......... .......... .......... 51% 26.1M 3s 2024-03-29T04:08:47.248 INFO:teuthology.orchestra.run.smithi097.stderr: 87100K .......... .......... .......... .......... .......... 51% 22.6M 3s 2024-03-29T04:08:47.248 INFO:teuthology.orchestra.run.smithi097.stderr: 87150K .......... .......... .......... .......... .......... 51% 36.9M 3s 2024-03-29T04:08:47.248 INFO:teuthology.orchestra.run.smithi097.stderr: 87200K .......... .......... .......... .......... .......... 51% 21.1M 3s 2024-03-29T04:08:47.248 INFO:teuthology.orchestra.run.smithi097.stderr: 87250K .......... .......... .......... .......... .......... 51% 24.5M 3s 2024-03-29T04:08:47.248 INFO:teuthology.orchestra.run.smithi097.stderr: 87300K .......... .......... .......... .......... .......... 51% 24.5M 3s 2024-03-29T04:08:47.248 INFO:teuthology.orchestra.run.smithi097.stderr: 87350K .......... .......... .......... .......... .......... 51% 112M 3s 2024-03-29T04:08:47.248 INFO:teuthology.orchestra.run.smithi097.stderr: 87400K .......... .......... .......... .......... .......... 51% 24.6M 3s 2024-03-29T04:08:47.248 INFO:teuthology.orchestra.run.smithi097.stderr: 87450K .......... .......... .......... .......... .......... 51% 24.4M 3s 2024-03-29T04:08:47.248 INFO:teuthology.orchestra.run.smithi097.stderr: 87500K .......... .......... .......... .......... .......... 51% 24.7M 3s 2024-03-29T04:08:47.248 INFO:teuthology.orchestra.run.smithi097.stderr: 87550K .......... .......... .......... .......... .......... 51% 110M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 87600K .......... .......... .......... .......... .......... 51% 26.3M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 87650K .......... .......... .......... .......... .......... 52% 24.2M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 87700K .......... .......... .......... .......... .......... 52% 25.8M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 87750K .......... .......... .......... .......... .......... 52% 85.3M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 87800K .......... .......... .......... .......... .......... 52% 21.3M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 87850K .......... .......... .......... .......... .......... 52% 23.8M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 87900K .......... .......... .......... .......... .......... 52% 63.8M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 87950K .......... .......... .......... .......... .......... 52% 19.8M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 88000K .......... .......... .......... .......... .......... 52% 27.3M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 88050K .......... .......... .......... .......... .......... 52% 33.8M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 88100K .......... .......... .......... .......... .......... 52% 24.4M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 88150K .......... .......... .......... .......... .......... 52% 41.0M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 88200K .......... .......... .......... .......... .......... 52% 24.3M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 88250K .......... .......... .......... .......... .......... 52% 2.48M 3s 2024-03-29T04:08:47.249 INFO:teuthology.orchestra.run.smithi097.stderr: 88300K .......... .......... .......... .......... .......... 52% 101M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 88350K .......... .......... .......... .......... .......... 52% 93.0M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 88400K .......... .......... .......... .......... .......... 52% 6.30M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 88450K .......... .......... .......... .......... .......... 52% 110M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 88500K .......... .......... .......... .......... .......... 52% 735K 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 88550K .......... .......... .......... .......... .......... 52% 109M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 88600K .......... .......... .......... .......... .......... 52% 116M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 88650K .......... .......... .......... .......... .......... 52% 17.0M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 88700K .......... .......... .......... .......... .......... 52% 262M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 88750K .......... .......... .......... .......... .......... 52% 48.9M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 88800K .......... .......... .......... .......... .......... 52% 21.6M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 88850K .......... .......... .......... .......... .......... 52% 111M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 88900K .......... .......... .......... .......... .......... 52% 23.0M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 88950K .......... .......... .......... .......... .......... 52% 109M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 89000K .......... .......... .......... .......... .......... 52% 62.2M 3s 2024-03-29T04:08:47.250 INFO:teuthology.orchestra.run.smithi097.stderr: 89050K .......... .......... .......... .......... .......... 52% 114M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89100K .......... .......... .......... .......... .......... 52% 71.7M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89150K .......... .......... .......... .......... .......... 52% 75.5M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89200K .......... .......... .......... .......... .......... 52% 24.7M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89250K .......... .......... .......... .......... .......... 52% 15.6M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89300K .......... .......... .......... .......... .......... 53% 103M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89350K .......... .......... .......... .......... .......... 53% 23.3M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89400K .......... .......... .......... .......... .......... 53% 11.0M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89450K .......... .......... .......... .......... .......... 53% 23.4M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89500K .......... .......... .......... .......... .......... 53% 115M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89550K .......... .......... .......... .......... .......... 53% 13.0M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89600K .......... .......... .......... .......... .......... 53% 22.6M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89650K .......... .......... .......... .......... .......... 53% 93.8M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89700K .......... .......... .......... .......... .......... 53% 83.7M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89750K .......... .......... .......... .......... .......... 53% 134M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89800K .......... .......... .......... .......... .......... 53% 80.3M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89850K .......... .......... .......... .......... .......... 53% 13.2M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89900K .......... .......... .......... .......... .......... 53% 13.1M 3s 2024-03-29T04:08:47.251 INFO:teuthology.orchestra.run.smithi097.stderr: 89950K .......... .......... .......... .......... .......... 53% 115M 3s 2024-03-29T04:08:47.252 INFO:teuthology.orchestra.run.smithi097.stderr: 90000K .......... .......... .......... .......... .......... 53% 11.5M 3s 2024-03-29T04:08:47.252 INFO:teuthology.orchestra.run.smithi097.stderr: 90050K .......... .......... .......... .......... .......... 53% 11.5M 3s 2024-03-29T04:08:47.252 INFO:teuthology.orchestra.run.smithi097.stderr: 90100K .......... .......... .......... .......... .......... 53% 22.9M 3s 2024-03-29T04:08:47.252 INFO:teuthology.orchestra.run.smithi097.stderr: 90150K .......... .......... .......... .......... .......... 53% 23.0M 3s 2024-03-29T04:08:47.252 INFO:teuthology.orchestra.run.smithi097.stderr: 90200K .......... .......... .......... .......... .......... 53% 114M 3s 2024-03-29T04:08:47.252 INFO:teuthology.orchestra.run.smithi097.stderr: 90250K .......... .......... .......... .......... .......... 53% 19.2M 3s 2024-03-29T04:08:47.252 INFO:teuthology.orchestra.run.smithi097.stderr: 90300K .......... .......... .......... .......... .......... 53% 21.6M 3s 2024-03-29T04:08:47.252 INFO:teuthology.orchestra.run.smithi097.stderr: 90350K .......... .......... .......... .......... .......... 53% 47.7M 3s 2024-03-29T04:08:47.252 INFO:teuthology.orchestra.run.smithi097.stderr: 90400K .......... .......... .......... .......... .......... 53% 111M 3s 2024-03-29T04:08:47.252 INFO:teuthology.orchestra.run.smithi097.stderr: 90450K .......... .......... .......... .......... .......... 53% 20.1M 3s 2024-03-29T04:08:47.252 INFO:teuthology.orchestra.run.smithi097.stderr: 90500K .......... .......... .......... .......... .......... 53% 26.5M 3s 2024-03-29T04:08:47.252 INFO:teuthology.orchestra.run.smithi097.stderr: 90550K .......... .......... .......... .......... .......... 53% 15.8M 3s 2024-03-29T04:08:47.252 INFO:teuthology.orchestra.run.smithi097.stderr: 90600K .......... .......... .......... .......... .......... 53% 24.4M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 90650K .......... .......... .......... .......... .......... 53% 48.8M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 90700K .......... .......... .......... .......... .......... 53% 18.2M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 90750K .......... .......... .......... .......... .......... 53% 22.8M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 90800K .......... .......... .......... .......... .......... 53% 17.8M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 90850K .......... .......... .......... .......... .......... 53% 29.9M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 90900K .......... .......... .......... .......... .......... 53% 22.2M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 90950K .......... .......... .......... .......... .......... 53% 107M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 91000K .......... .......... .......... .......... .......... 54% 23.7M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 91050K .......... .......... .......... .......... .......... 54% 77.5M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 91100K .......... .......... .......... .......... .......... 54% 29.7M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 91150K .......... .......... .......... .......... .......... 54% 39.0M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 91200K .......... .......... .......... .......... .......... 54% 112M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 91250K .......... .......... .......... .......... .......... 54% 98.1M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 91300K .......... .......... .......... .......... .......... 54% 33.4M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 91350K .......... .......... .......... .......... .......... 54% 87.3M 3s 2024-03-29T04:08:47.253 INFO:teuthology.orchestra.run.smithi097.stderr: 91400K .......... .......... .......... .......... .......... 54% 99.2M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 91450K .......... .......... .......... .......... .......... 54% 106M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 91500K .......... .......... .......... .......... .......... 54% 68.0M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 91550K .......... .......... .......... .......... .......... 54% 106M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 91600K .......... .......... .......... .......... .......... 54% 110M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 91650K .......... .......... .......... .......... .......... 54% 62.0M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 91700K .......... .......... .......... .......... .......... 54% 67.8M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 91750K .......... .......... .......... .......... .......... 54% 23.8M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 91800K .......... .......... .......... .......... .......... 54% 33.9M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 91850K .......... .......... .......... .......... .......... 54% 24.8M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 91900K .......... .......... .......... .......... .......... 54% 24.0M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 91950K .......... .......... .......... .......... .......... 54% 55.7M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 92000K .......... .......... .......... .......... .......... 54% 26.1M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 92050K .......... .......... .......... .......... .......... 54% 30.7M 3s 2024-03-29T04:08:47.254 INFO:teuthology.orchestra.run.smithi097.stderr: 92100K .......... .......... .......... .......... .......... 54% 25.4M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92150K .......... .......... .......... .......... .......... 54% 25.6M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92200K .......... .......... .......... .......... .......... 54% 44.2M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92250K .......... .......... .......... .......... .......... 54% 24.0M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92300K .......... .......... .......... .......... .......... 54% 42.4M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92350K .......... .......... .......... .......... .......... 54% 21.9M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92400K .......... .......... .......... .......... .......... 54% 27.5M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92450K .......... .......... .......... .......... .......... 54% 41.9M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92500K .......... .......... .......... .......... .......... 54% 30.5M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92550K .......... .......... .......... .......... .......... 54% 21.0M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92600K .......... .......... .......... .......... .......... 54% 31.1M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92650K .......... .......... .......... .......... .......... 54% 44.3M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92700K .......... .......... .......... .......... .......... 55% 27.6M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92750K .......... .......... .......... .......... .......... 55% 24.0M 3s 2024-03-29T04:08:47.255 INFO:teuthology.orchestra.run.smithi097.stderr: 92800K .......... .......... .......... .......... .......... 55% 31.9M 3s 2024-03-29T04:08:47.256 INFO:teuthology.orchestra.run.smithi097.stderr: 92850K .......... .......... .......... .......... .......... 55% 22.3M 3s 2024-03-29T04:08:47.256 INFO:teuthology.orchestra.run.smithi097.stderr: 92900K .......... .......... .......... .......... .......... 55% 39.6M 3s 2024-03-29T04:08:47.256 INFO:teuthology.orchestra.run.smithi097.stderr: 92950K .......... .......... .......... .......... .......... 55% 35.4M 3s 2024-03-29T04:08:47.256 INFO:teuthology.orchestra.run.smithi097.stderr: 93000K .......... .......... .......... .......... .......... 55% 29.2M 3s 2024-03-29T04:08:47.256 INFO:teuthology.orchestra.run.smithi097.stderr: 93050K .......... .......... .......... .......... .......... 55% 20.0M 3s 2024-03-29T04:08:47.256 INFO:teuthology.orchestra.run.smithi097.stderr: 93100K .......... .......... .......... .......... .......... 55% 26.8M 3s 2024-03-29T04:08:47.256 INFO:teuthology.orchestra.run.smithi097.stderr: 93150K .......... .......... .......... .......... .......... 55% 42.7M 3s 2024-03-29T04:08:47.256 INFO:teuthology.orchestra.run.smithi097.stderr: 93200K .......... .......... .......... .......... .......... 55% 36.5M 3s 2024-03-29T04:08:47.256 INFO:teuthology.orchestra.run.smithi097.stderr: 93250K .......... .......... .......... .......... .......... 55% 27.2M 3s 2024-03-29T04:08:47.256 INFO:teuthology.orchestra.run.smithi097.stderr: 93300K .......... .......... .......... .......... .......... 55% 21.8M 3s 2024-03-29T04:08:47.256 INFO:teuthology.orchestra.run.smithi097.stderr: 93350K .......... .......... .......... .......... .......... 55% 26.6M 3s 2024-03-29T04:08:47.256 INFO:teuthology.orchestra.run.smithi097.stderr: 93400K .......... .......... .......... .......... .......... 55% 36.0M 3s 2024-03-29T04:08:47.256 INFO:teuthology.orchestra.run.smithi097.stderr: 93450K .......... .......... .......... .......... .......... 55% 36.9M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 93500K .......... .......... .......... .......... .......... 55% 26.3M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 93550K .......... .......... .......... .......... .......... 55% 35.4M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 93600K .......... .......... .......... .......... .......... 55% 22.6M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 93650K .......... .......... .......... .......... .......... 55% 25.7M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 93700K .......... .......... .......... .......... .......... 55% 36.5M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 93750K .......... .......... .......... .......... .......... 55% 24.5M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 93800K .......... .......... .......... .......... .......... 55% 32.6M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 93850K .......... .......... .......... .......... .......... 55% 33.6M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 93900K .......... .......... .......... .......... .......... 55% 19.6M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 93950K .......... .......... .......... .......... .......... 55% 34.8M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 94000K .......... .......... .......... .......... .......... 55% 26.4M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 94050K .......... .......... .......... .......... .......... 55% 32.5M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 94100K .......... .......... .......... .......... .......... 55% 31.2M 3s 2024-03-29T04:08:47.257 INFO:teuthology.orchestra.run.smithi097.stderr: 94150K .......... .......... .......... .......... .......... 55% 15.4M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94200K .......... .......... .......... .......... .......... 55% 22.7M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94250K .......... .......... .......... .......... .......... 55% 62.3M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94300K .......... .......... .......... .......... .......... 55% 61.3M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94350K .......... .......... .......... .......... .......... 56% 33.9M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94400K .......... .......... .......... .......... .......... 56% 29.5M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94450K .......... .......... .......... .......... .......... 56% 30.9M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94500K .......... .......... .......... .......... .......... 56% 38.8M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94550K .......... .......... .......... .......... .......... 56% 21.3M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94600K .......... .......... .......... .......... .......... 56% 24.6M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94650K .......... .......... .......... .......... .......... 56% 23.7M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94700K .......... .......... .......... .......... .......... 56% 24.5M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94750K .......... .......... .......... .......... .......... 56% 109M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94800K .......... .......... .......... .......... .......... 56% 23.6M 3s 2024-03-29T04:08:47.258 INFO:teuthology.orchestra.run.smithi097.stderr: 94850K .......... .......... .......... .......... .......... 56% 24.4M 3s 2024-03-29T04:08:47.259 INFO:teuthology.orchestra.run.smithi097.stderr: 94900K .......... .......... .......... .......... .......... 56% 23.7M 3s 2024-03-29T04:08:47.259 INFO:teuthology.orchestra.run.smithi097.stderr: 94950K .......... .......... .......... .......... .......... 56% 25.0M 3s 2024-03-29T04:08:47.259 INFO:teuthology.orchestra.run.smithi097.stderr: 95000K .......... .......... .......... .......... .......... 56% 109M 3s 2024-03-29T04:08:47.259 INFO:teuthology.orchestra.run.smithi097.stderr: 95050K .......... .......... .......... .......... .......... 56% 24.0M 3s 2024-03-29T04:08:47.259 INFO:teuthology.orchestra.run.smithi097.stderr: 95100K .......... .......... .......... .......... .......... 56% 24.6M 3s 2024-03-29T04:08:47.259 INFO:teuthology.orchestra.run.smithi097.stderr: 95150K .......... .......... .......... .......... .......... 56% 22.3M 3s 2024-03-29T04:08:47.259 INFO:teuthology.orchestra.run.smithi097.stderr: 95200K .......... .......... .......... .......... .......... 56% 112M 3s 2024-03-29T04:08:47.259 INFO:teuthology.orchestra.run.smithi097.stderr: 95250K .......... .......... .......... .......... .......... 56% 23.7M 3s 2024-03-29T04:08:47.259 INFO:teuthology.orchestra.run.smithi097.stderr: 95300K .......... .......... .......... .......... .......... 56% 22.3M 3s 2024-03-29T04:08:47.259 INFO:teuthology.orchestra.run.smithi097.stderr: 95350K .......... .......... .......... .......... .......... 56% 45.8M 3s 2024-03-29T04:08:47.259 INFO:teuthology.orchestra.run.smithi097.stderr: 95400K .......... .......... .......... .......... .......... 56% 32.4M 3s 2024-03-29T04:08:47.259 INFO:teuthology.orchestra.run.smithi097.stderr: 95450K .......... .......... .......... .......... .......... 56% 26.8M 3s 2024-03-29T04:08:47.259 INFO:teuthology.orchestra.run.smithi097.stderr: 95500K .......... .......... .......... .......... .......... 56% 47.4M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 95550K .......... .......... .......... .......... .......... 56% 22.4M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 95600K .......... .......... .......... .......... .......... 56% 38.6M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 95650K .......... .......... .......... .......... .......... 56% 18.2M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 95700K .......... .......... .......... .......... .......... 56% 33.7M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 95750K .......... .......... .......... .......... .......... 56% 66.5M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 95800K .......... .......... .......... .......... .......... 56% 22.9M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 95850K .......... .......... .......... .......... .......... 56% 17.7M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 95900K .......... .......... .......... .......... .......... 56% 34.2M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 95950K .......... .......... .......... .......... .......... 56% 87.7M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 96000K .......... .......... .......... .......... .......... 56% 30.0M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 96050K .......... .......... .......... .......... .......... 57% 23.3M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 96100K .......... .......... .......... .......... .......... 57% 26.0M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 96150K .......... .......... .......... .......... .......... 57% 37.0M 3s 2024-03-29T04:08:47.260 INFO:teuthology.orchestra.run.smithi097.stderr: 96200K .......... .......... .......... .......... .......... 57% 28.5M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96250K .......... .......... .......... .......... .......... 57% 31.2M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96300K .......... .......... .......... .......... .......... 57% 26.9M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96350K .......... .......... .......... .......... .......... 57% 23.0M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96400K .......... .......... .......... .......... .......... 57% 24.0M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96450K .......... .......... .......... .......... .......... 57% 26.3M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96500K .......... .......... .......... .......... .......... 57% 25.2M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96550K .......... .......... .......... .......... .......... 57% 111M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96600K .......... .......... .......... .......... .......... 57% 23.1M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96650K .......... .......... .......... .......... .......... 57% 22.7M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96700K .......... .......... .......... .......... .......... 57% 34.9M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96750K .......... .......... .......... .......... .......... 57% 24.5M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96800K .......... .......... .......... .......... .......... 57% 51.4M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96850K .......... .......... .......... .......... .......... 57% 26.7M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96900K .......... .......... .......... .......... .......... 57% 23.2M 3s 2024-03-29T04:08:47.261 INFO:teuthology.orchestra.run.smithi097.stderr: 96950K .......... .......... .......... .......... .......... 57% 29.2M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97000K .......... .......... .......... .......... .......... 57% 25.2M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97050K .......... .......... .......... .......... .......... 57% 110M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97100K .......... .......... .......... .......... .......... 57% 25.3M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97150K .......... .......... .......... .......... .......... 57% 24.8M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97200K .......... .......... .......... .......... .......... 57% 25.3M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97250K .......... .......... .......... .......... .......... 57% 25.0M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97300K .......... .......... .......... .......... .......... 57% 59.8M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97350K .......... .......... .......... .......... .......... 57% 20.6M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97400K .......... .......... .......... .......... .......... 57% 32.6M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97450K .......... .......... .......... .......... .......... 57% 28.3M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97500K .......... .......... .......... .......... .......... 57% 40.6M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97550K .......... .......... .......... .......... .......... 57% 27.4M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97600K .......... .......... .......... .......... .......... 57% 29.3M 3s 2024-03-29T04:08:47.262 INFO:teuthology.orchestra.run.smithi097.stderr: 97650K .......... .......... .......... .......... .......... 57% 29.5M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 97700K .......... .......... .......... .......... .......... 57% 24.2M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 97750K .......... .......... .......... .......... .......... 58% 24.6M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 97800K .......... .......... .......... .......... .......... 58% 65.5M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 97850K .......... .......... .......... .......... .......... 58% 21.7M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 97900K .......... .......... .......... .......... .......... 58% 33.6M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 97950K .......... .......... .......... .......... .......... 58% 23.2M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 98000K .......... .......... .......... .......... .......... 58% 42.2M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 98050K .......... .......... .......... .......... .......... 58% 29.3M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 98100K .......... .......... .......... .......... .......... 58% 27.0M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 98150K .......... .......... .......... .......... .......... 58% 25.1M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 98200K .......... .......... .......... .......... .......... 58% 23.9M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 98250K .......... .......... .......... .......... .......... 58% 27.3M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 98300K .......... .......... .......... .......... .......... 58% 46.3M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 98350K .......... .......... .......... .......... .......... 58% 27.9M 3s 2024-03-29T04:08:47.263 INFO:teuthology.orchestra.run.smithi097.stderr: 98400K .......... .......... .......... .......... .......... 58% 31.9M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 98450K .......... .......... .......... .......... .......... 58% 26.8M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 98500K .......... .......... .......... .......... .......... 58% 19.2M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 98550K .......... .......... .......... .......... .......... 58% 72.2M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 98600K .......... .......... .......... .......... .......... 58% 24.1M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 98650K .......... .......... .......... .......... .......... 58% 34.0M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 98700K .......... .......... .......... .......... .......... 58% 24.8M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 98750K .......... .......... .......... .......... .......... 58% 24.5M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 98800K .......... .......... .......... .......... .......... 58% 51.4M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 98850K .......... .......... .......... .......... .......... 58% 23.7M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 98900K .......... .......... .......... .......... .......... 58% 25.0M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 98950K .......... .......... .......... .......... .......... 58% 33.7M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 99000K .......... .......... .......... .......... .......... 58% 23.8M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 99050K .......... .......... .......... .......... .......... 58% 40.8M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 99100K .......... .......... .......... .......... .......... 58% 24.7M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 99150K .......... .......... .......... .......... .......... 58% 43.6M 3s 2024-03-29T04:08:47.264 INFO:teuthology.orchestra.run.smithi097.stderr: 99200K .......... .......... .......... .......... .......... 58% 24.2M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99250K .......... .......... .......... .......... .......... 58% 24.2M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99300K .......... .......... .......... .......... .......... 58% 45.6M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99350K .......... .......... .......... .......... .......... 58% 24.7M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99400K .......... .......... .......... .......... .......... 58% 33.9M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99450K .......... .......... .......... .......... .......... 59% 24.5M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99500K .......... .......... .......... .......... .......... 59% 24.8M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99550K .......... .......... .......... .......... .......... 59% 49.9M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99600K .......... .......... .......... .......... .......... 59% 23.5M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99650K .......... .......... .......... .......... .......... 59% 29.9M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99700K .......... .......... .......... .......... .......... 59% 29.9M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99750K .......... .......... .......... .......... .......... 59% 27.3M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99800K .......... .......... .......... .......... .......... 59% 37.9M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99850K .......... .......... .......... .......... .......... 59% 25.8M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99900K .......... .......... .......... .......... .......... 59% 31.0M 3s 2024-03-29T04:08:47.265 INFO:teuthology.orchestra.run.smithi097.stderr: 99950K .......... .......... .......... .......... .......... 59% 23.0M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100000K .......... .......... .......... .......... .......... 59% 47.1M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100050K .......... .......... .......... .......... .......... 59% 24.7M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100100K .......... .......... .......... .......... .......... 59% 25.8M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100150K .......... .......... .......... .......... .......... 59% 28.1M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100200K .......... .......... .......... .......... .......... 59% 28.2M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100250K .......... .......... .......... .......... .......... 59% 40.4M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100300K .......... .......... .......... .......... .......... 59% 30.6M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100350K .......... .......... .......... .......... .......... 59% 22.4M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100400K .......... .......... .......... .......... .......... 59% 24.6M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100450K .......... .......... .......... .......... .......... 59% 26.4M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100500K .......... .......... .......... .......... .......... 59% 73.0M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100550K .......... .......... .......... .......... .......... 59% 24.7M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100600K .......... .......... .......... .......... .......... 59% 25.3M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100650K .......... .......... .......... .......... .......... 59% 24.5M 3s 2024-03-29T04:08:47.266 INFO:teuthology.orchestra.run.smithi097.stderr:100700K .......... .......... .......... .......... .......... 59% 18.3M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:100750K .......... .......... .......... .......... .......... 59% 45.7M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:100800K .......... .......... .......... .......... .......... 59% 98.5M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:100850K .......... .......... .......... .......... .......... 59% 23.7M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:100900K .......... .......... .......... .......... .......... 59% 26.9M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:100950K .......... .......... .......... .......... .......... 59% 24.5M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:101000K .......... .......... .......... .......... .......... 59% 53.4M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:101050K .......... .......... .......... .......... .......... 59% 31.5M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:101100K .......... .......... .......... .......... .......... 60% 25.6M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:101150K .......... .......... .......... .......... .......... 60% 20.3M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:101200K .......... .......... .......... .......... .......... 60% 33.2M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:101250K .......... .......... .......... .......... .......... 60% 47.7M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:101300K .......... .......... .......... .......... .......... 60% 27.9M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:101350K .......... .......... .......... .......... .......... 60% 28.1M 3s 2024-03-29T04:08:47.267 INFO:teuthology.orchestra.run.smithi097.stderr:101400K .......... .......... .......... .......... .......... 60% 22.6M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:101450K .......... .......... .......... .......... .......... 60% 26.4M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:101500K .......... .......... .......... .......... .......... 60% 51.9M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:101550K .......... .......... .......... .......... .......... 60% 32.0M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:101600K .......... .......... .......... .......... .......... 60% 25.8M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:101650K .......... .......... .......... .......... .......... 60% 20.7M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:101700K .......... .......... .......... .......... .......... 60% 27.2M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:101750K .......... .......... .......... .......... .......... 60% 45.5M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:101800K .......... .......... .......... .......... .......... 60% 22.4M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:101850K .......... .......... .......... .......... .......... 60% 35.5M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:101900K .......... .......... .......... .......... .......... 60% 25.3M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:101950K .......... .......... .......... .......... .......... 60% 26.7M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:102000K .......... .......... .......... .......... .......... 60% 45.3M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:102050K .......... .......... .......... .......... .......... 60% 21.3M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:102100K .......... .......... .......... .......... .......... 60% 42.7M 3s 2024-03-29T04:08:47.268 INFO:teuthology.orchestra.run.smithi097.stderr:102150K .......... .......... .......... .......... .......... 60% 18.9M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102200K .......... .......... .......... .......... .......... 60% 26.1M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102250K .......... .......... .......... .......... .......... 60% 15.2M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102300K .......... .......... .......... .......... .......... 60% 42.5M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102350K .......... .......... .......... .......... .......... 60% 46.9M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102400K .......... .......... .......... .......... .......... 60% 41.7M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102450K .......... .......... .......... .......... .......... 60% 75.7M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102500K .......... .......... .......... .......... .......... 60% 23.7M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102550K .......... .......... .......... .......... .......... 60% 28.4M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102600K .......... .......... .......... .......... .......... 60% 37.7M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102650K .......... .......... .......... .......... .......... 60% 24.1M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102700K .......... .......... .......... .......... .......... 60% 34.9M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102750K .......... .......... .......... .......... .......... 60% 21.8M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102800K .......... .......... .......... .......... .......... 61% 31.1M 3s 2024-03-29T04:08:47.269 INFO:teuthology.orchestra.run.smithi097.stderr:102850K .......... .......... .......... .......... .......... 61% 42.9M 3s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:102900K .......... .......... .......... .......... .......... 61% 30.0M 3s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:102950K .......... .......... .......... .......... .......... 61% 28.9M 3s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:103000K .......... .......... .......... .......... .......... 61% 23.5M 3s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:103050K .......... .......... .......... .......... .......... 61% 24.6M 3s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:103100K .......... .......... .......... .......... .......... 61% 44.3M 3s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:103150K .......... .......... .......... .......... .......... 61% 32.0M 3s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:103200K .......... .......... .......... .......... .......... 61% 26.7M 3s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:103250K .......... .......... .......... .......... .......... 61% 25.0M 3s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:103300K .......... .......... .......... .......... .......... 61% 21.3M 3s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:103350K .......... .......... .......... .......... .......... 61% 75.9M 3s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:103400K .......... .......... .......... .......... .......... 61% 19.2M 3s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:103450K .......... .......... .......... .......... .......... 61% 32.4M 3s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:103500K .......... .......... .......... .......... .......... 61% 30.9M 2s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:103550K .......... .......... .......... .......... .......... 61% 26.4M 2s 2024-03-29T04:08:47.270 INFO:teuthology.orchestra.run.smithi097.stderr:103600K .......... .......... .......... .......... .......... 61% 59.5M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:103650K .......... .......... .......... .......... .......... 61% 22.1M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:103700K .......... .......... .......... .......... .......... 61% 35.7M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:103750K .......... .......... .......... .......... .......... 61% 9.13M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:103800K .......... .......... .......... .......... .......... 61% 135M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:103850K .......... .......... .......... .......... .......... 61% 208M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:103900K .......... .......... .......... .......... .......... 61% 40.4M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:103950K .......... .......... .......... .......... .......... 61% 36.2M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:104000K .......... .......... .......... .......... .......... 61% 26.4M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:104050K .......... .......... .......... .......... .......... 61% 24.6M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:104100K .......... .......... .......... .......... .......... 61% 50.7M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:104150K .......... .......... .......... .......... .......... 61% 26.6M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:104200K .......... .......... .......... .......... .......... 61% 23.0M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:104250K .......... .......... .......... .......... .......... 61% 23.9M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:104300K .......... .......... .......... .......... .......... 61% 33.3M 2s 2024-03-29T04:08:47.271 INFO:teuthology.orchestra.run.smithi097.stderr:104350K .......... .......... .......... .......... .......... 61% 52.3M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:104400K .......... .......... .......... .......... .......... 61% 23.9M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:104450K .......... .......... .......... .......... .......... 61% 27.2M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:104500K .......... .......... .......... .......... .......... 62% 26.5M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:104550K .......... .......... .......... .......... .......... 62% 26.7M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:104600K .......... .......... .......... .......... .......... 62% 46.9M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:104650K .......... .......... .......... .......... .......... 62% 24.7M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:104700K .......... .......... .......... .......... .......... 62% 38.3M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:104750K .......... .......... .......... .......... .......... 62% 24.1M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:104800K .......... .......... .......... .......... .......... 62% 23.3M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:104850K .......... .......... .......... .......... .......... 62% 47.6M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:104900K .......... .......... .......... .......... .......... 62% 25.2M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:104950K .......... .......... .......... .......... .......... 62% 34.4M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:105000K .......... .......... .......... .......... .......... 62% 21.4M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:105050K .......... .......... .......... .......... .......... 62% 27.9M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:105100K .......... .......... .......... .......... .......... 62% 38.6M 2s 2024-03-29T04:08:47.272 INFO:teuthology.orchestra.run.smithi097.stderr:105150K .......... .......... .......... .......... .......... 62% 22.8M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105200K .......... .......... .......... .......... .......... 62% 38.5M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105250K .......... .......... .......... .......... .......... 62% 29.6M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105300K .......... .......... .......... .......... .......... 62% 24.5M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105350K .......... .......... .......... .......... .......... 62% 53.1M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105400K .......... .......... .......... .......... .......... 62% 27.0M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105450K .......... .......... .......... .......... .......... 62% 29.3M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105500K .......... .......... .......... .......... .......... 62% 24.9M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105550K .......... .......... .......... .......... .......... 62% 22.0M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105600K .......... .......... .......... .......... .......... 62% 30.2M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105650K .......... .......... .......... .......... .......... 62% 23.6M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105700K .......... .......... .......... .......... .......... 62% 36.4M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105750K .......... .......... .......... .......... .......... 62% 37.4M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105800K .......... .......... .......... .......... .......... 62% 837K 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105850K .......... .......... .......... .......... .......... 62% 108M 2s 2024-03-29T04:08:47.273 INFO:teuthology.orchestra.run.smithi097.stderr:105900K .......... .......... .......... .......... .......... 62% 115M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:105950K .......... .......... .......... .......... .......... 62% 113M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106000K .......... .......... .......... .......... .......... 62% 111M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106050K .......... .......... .......... .......... .......... 62% 793K 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106100K .......... .......... .......... .......... .......... 62% 76.9M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106150K .......... .......... .......... .......... .......... 63% 85.7M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106200K .......... .......... .......... .......... .......... 63% 112M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106250K .......... .......... .......... .......... .......... 63% 109M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106300K .......... .......... .......... .......... .......... 63% 3.22M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106350K .......... .......... .......... .......... .......... 63% 116M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106400K .......... .......... .......... .......... .......... 63% 112M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106450K .......... .......... .......... .......... .......... 63% 9.93M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106500K .......... .......... .......... .......... .......... 63% 110M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106550K .......... .......... .......... .......... .......... 63% 6.54M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106600K .......... .......... .......... .......... .......... 63% 115M 2s 2024-03-29T04:08:47.274 INFO:teuthology.orchestra.run.smithi097.stderr:106650K .......... .......... .......... .......... .......... 63% 111M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:106700K .......... .......... .......... .......... .......... 63% 49.0M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:106750K .......... .......... .......... .......... .......... 63% 114M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:106800K .......... .......... .......... .......... .......... 63% 108M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:106850K .......... .......... .......... .......... .......... 63% 49.1M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:106900K .......... .......... .......... .......... .......... 63% 96.3M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:106950K .......... .......... .......... .......... .......... 63% 40.3M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:107000K .......... .......... .......... .......... .......... 63% 27.3M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:107050K .......... .......... .......... .......... .......... 63% 111M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:107100K .......... .......... .......... .......... .......... 63% 2.55M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:107150K .......... .......... .......... .......... .......... 63% 110M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:107200K .......... .......... .......... .......... .......... 63% 1.40M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:107250K .......... .......... .......... .......... .......... 63% 112M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:107300K .......... .......... .......... .......... .......... 63% 117M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:107350K .......... .......... .......... .......... .......... 63% 110M 2s 2024-03-29T04:08:47.275 INFO:teuthology.orchestra.run.smithi097.stderr:107400K .......... .......... .......... .......... .......... 63% 109M 2s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:107450K .......... .......... .......... .......... .......... 63% 903K 2s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:107500K .......... .......... .......... .......... .......... 63% 118M 2s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:107550K .......... .......... .......... .......... .......... 63% 111M 2s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:107600K .......... .......... .......... .......... .......... 63% 2.64M 2s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:107650K .......... .......... .......... .......... .......... 63% 109M 2s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:107700K .......... .......... .......... .......... .......... 63% 116M 2s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:107750K .......... .......... .......... .......... .......... 63% 111M 2s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:107800K .......... .......... .......... .......... .......... 63% 112M 2s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:107850K .......... .......... .......... .......... .......... 64% 497K 3s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:107900K .......... .......... .......... .......... .......... 64% 103M 2s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:107950K .......... .......... .......... .......... .......... 64% 112M 2s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:108000K .......... .......... .......... .......... .......... 64% 2.03M 3s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:108050K .......... .......... .......... .......... .......... 64% 114M 3s 2024-03-29T04:08:47.276 INFO:teuthology.orchestra.run.smithi097.stderr:108100K .......... .......... .......... .......... .......... 64% 5.51M 3s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108150K .......... .......... .......... .......... .......... 64% 116M 3s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108200K .......... .......... .......... .......... .......... 64% 104M 2s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108250K .......... .......... .......... .......... .......... 64% 102M 2s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108300K .......... .......... .......... .......... .......... 64% 125M 2s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108350K .......... .......... .......... .......... .......... 64% 817K 3s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108400K .......... .......... .......... .......... .......... 64% 108M 3s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108450K .......... .......... .......... .......... .......... 64% 110M 3s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108500K .......... .......... .......... .......... .......... 64% 3.67M 3s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108550K .......... .......... .......... .......... .......... 64% 108M 3s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108600K .......... .......... .......... .......... .......... 64% 109M 3s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108650K .......... .......... .......... .......... .......... 64% 120M 3s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108700K .......... .......... .......... .......... .......... 64% 112M 3s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108750K .......... .......... .......... .......... .......... 64% 19.3M 3s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108800K .......... .......... .......... .......... .......... 64% 114M 3s 2024-03-29T04:08:47.277 INFO:teuthology.orchestra.run.smithi097.stderr:108850K .......... .......... .......... .......... .......... 64% 110M 3s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:108900K .......... .......... .......... .......... .......... 64% 110M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:108950K .......... .......... .......... .......... .......... 64% 33.0M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:109000K .......... .......... .......... .......... .......... 64% 20.4M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:109050K .......... .......... .......... .......... .......... 64% 27.5M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:109100K .......... .......... .......... .......... .......... 64% 110M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:109150K .......... .......... .......... .......... .......... 64% 21.2M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:109200K .......... .......... .......... .......... .......... 64% 28.1M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:109250K .......... .......... .......... .......... .......... 64% 24.1M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:109300K .......... .......... .......... .......... .......... 64% 23.8M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:109350K .......... .......... .......... .......... .......... 64% 115M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:109400K .......... .......... .......... .......... .......... 64% 23.7M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:109450K .......... .......... .......... .......... .......... 64% 24.7M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:109500K .......... .......... .......... .......... .......... 64% 24.1M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:109550K .......... .......... .......... .......... .......... 65% 109M 2s 2024-03-29T04:08:47.278 INFO:teuthology.orchestra.run.smithi097.stderr:109600K .......... .......... .......... .......... .......... 65% 24.7M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:109650K .......... .......... .......... .......... .......... 65% 23.4M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:109700K .......... .......... .......... .......... .......... 65% 24.4M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:109750K .......... .......... .......... .......... .......... 65% 23.8M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:109800K .......... .......... .......... .......... .......... 65% 112M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:109850K .......... .......... .......... .......... .......... 65% 24.9M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:109900K .......... .......... .......... .......... .......... 65% 23.7M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:109950K .......... .......... .......... .......... .......... 65% 25.0M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:110000K .......... .......... .......... .......... .......... 65% 111M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:110050K .......... .......... .......... .......... .......... 65% 23.2M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:110100K .......... .......... .......... .......... .......... 65% 24.8M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:110150K .......... .......... .......... .......... .......... 65% 23.8M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:110200K .......... .......... .......... .......... .......... 65% 24.5M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:110250K .......... .......... .......... .......... .......... 65% 93.6M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:110300K .......... .......... .......... .......... .......... 65% 24.5M 2s 2024-03-29T04:08:47.279 INFO:teuthology.orchestra.run.smithi097.stderr:110350K .......... .......... .......... .......... .......... 65% 24.8M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:110400K .......... .......... .......... .......... .......... 65% 23.7M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:110450K .......... .......... .......... .......... .......... 65% 24.4M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:110500K .......... .......... .......... .......... .......... 65% 117M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:110550K .......... .......... .......... .......... .......... 65% 23.4M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:110600K .......... .......... .......... .......... .......... 65% 24.7M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:110650K .......... .......... .......... .......... .......... 65% 106M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:110700K .......... .......... .......... .......... .......... 65% 22.7M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:110750K .......... .......... .......... .......... .......... 65% 23.9M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:110800K .......... .......... .......... .......... .......... 65% 25.3M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:110850K .......... .......... .......... .......... .......... 65% 21.7M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:110900K .......... .......... .......... .......... .......... 65% 26.7M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:110950K .......... .......... .......... .......... .......... 65% 102M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:111000K .......... .......... .......... .......... .......... 65% 26.2M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:111050K .......... .......... .......... .......... .......... 65% 27.0M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:111100K .......... .......... .......... .......... .......... 65% 21.9M 2s 2024-03-29T04:08:47.280 INFO:teuthology.orchestra.run.smithi097.stderr:111150K .......... .......... .......... .......... .......... 65% 25.0M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111200K .......... .......... .......... .......... .......... 65% 117M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111250K .......... .......... .......... .......... .......... 66% 28.4M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111300K .......... .......... .......... .......... .......... 66% 38.3M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111350K .......... .......... .......... .......... .......... 66% 23.7M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111400K .......... .......... .......... .......... .......... 66% 22.6M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111450K .......... .......... .......... .......... .......... 66% 60.9M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111500K .......... .......... .......... .......... .......... 66% 28.6M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111550K .......... .......... .......... .......... .......... 66% 22.6M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111600K .......... .......... .......... .......... .......... 66% 23.3M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111650K .......... .......... .......... .......... .......... 66% 23.4M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111700K .......... .......... .......... .......... .......... 66% 119M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111750K .......... .......... .......... .......... .......... 66% 25.1M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111800K .......... .......... .......... .......... .......... 66% 23.1M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111850K .......... .......... .......... .......... .......... 66% 24.4M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111900K .......... .......... .......... .......... .......... 66% 25.4M 2s 2024-03-29T04:08:47.281 INFO:teuthology.orchestra.run.smithi097.stderr:111950K .......... .......... .......... .......... .......... 66% 97.0M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112000K .......... .......... .......... .......... .......... 66% 25.9M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112050K .......... .......... .......... .......... .......... 66% 25.3M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112100K .......... .......... .......... .......... .......... 66% 23.6M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112150K .......... .......... .......... .......... .......... 66% 106M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112200K .......... .......... .......... .......... .......... 66% 25.4M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112250K .......... .......... .......... .......... .......... 66% 23.9M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112300K .......... .......... .......... .......... .......... 66% 25.6M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112350K .......... .......... .......... .......... .......... 66% 22.8M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112400K .......... .......... .......... .......... .......... 66% 90.4M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112450K .......... .......... .......... .......... .......... 66% 28.2M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112500K .......... .......... .......... .......... .......... 66% 23.7M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112550K .......... .......... .......... .......... .......... 66% 24.5M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112600K .......... .......... .......... .......... .......... 66% 44.9M 2s 2024-03-29T04:08:47.282 INFO:teuthology.orchestra.run.smithi097.stderr:112650K .......... .......... .......... .......... .......... 66% 24.8M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:112700K .......... .......... .......... .......... .......... 66% 39.1M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:112750K .......... .......... .......... .......... .......... 66% 24.5M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:112800K .......... .......... .......... .......... .......... 66% 37.3M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:112850K .......... .......... .......... .......... .......... 66% 23.5M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:112900K .......... .......... .......... .......... .......... 67% 24.4M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:112950K .......... .......... .......... .......... .......... 67% 49.6M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:113000K .......... .......... .......... .......... .......... 67% 28.5M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:113050K .......... .......... .......... .......... .......... 67% 29.7M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:113100K .......... .......... .......... .......... .......... 67% 24.8M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:113150K .......... .......... .......... .......... .......... 67% 23.0M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:113200K .......... .......... .......... .......... .......... 67% 60.2M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:113250K .......... .......... .......... .......... .......... 67% 28.1M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:113300K .......... .......... .......... .......... .......... 67% 25.3M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:113350K .......... .......... .......... .......... .......... 67% 25.7M 2s 2024-03-29T04:08:47.283 INFO:teuthology.orchestra.run.smithi097.stderr:113400K .......... .......... .......... .......... .......... 67% 34.0M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:113450K .......... .......... .......... .......... .......... 67% 31.1M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:113500K .......... .......... .......... .......... .......... 67% 28.2M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:113550K .......... .......... .......... .......... .......... 67% 30.5M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:113600K .......... .......... .......... .......... .......... 67% 24.4M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:113650K .......... .......... .......... .......... .......... 67% 24.7M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:113700K .......... .......... .......... .......... .......... 67% 56.8M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:113750K .......... .......... .......... .......... .......... 67% 24.0M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:113800K .......... .......... .......... .......... .......... 67% 25.1M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:113850K .......... .......... .......... .......... .......... 67% 31.7M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:113900K .......... .......... .......... .......... .......... 67% 24.6M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:113950K .......... .......... .......... .......... .......... 67% 56.9M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:114000K .......... .......... .......... .......... .......... 67% 23.4M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:114050K .......... .......... .......... .......... .......... 67% 23.9M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:114100K .......... .......... .......... .......... .......... 67% 35.0M 2s 2024-03-29T04:08:47.284 INFO:teuthology.orchestra.run.smithi097.stderr:114150K .......... .......... .......... .......... .......... 67% 24.6M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114200K .......... .......... .......... .......... .......... 67% 43.0M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114250K .......... .......... .......... .......... .......... 67% 25.3M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114300K .......... .......... .......... .......... .......... 67% 23.5M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114350K .......... .......... .......... .......... .......... 67% 35.9M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114400K .......... .......... .......... .......... .......... 67% 24.1M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114450K .......... .......... .......... .......... .......... 67% 51.8M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114500K .......... .......... .......... .......... .......... 67% 25.1M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114550K .......... .......... .......... .......... .......... 67% 29.3M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114600K .......... .......... .......... .......... .......... 68% 21.3M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114650K .......... .......... .......... .......... .......... 68% 32.3M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114700K .......... .......... .......... .......... .......... 68% 45.9M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114750K .......... .......... .......... .......... .......... 68% 24.0M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114800K .......... .......... .......... .......... .......... 68% 25.9M 2s 2024-03-29T04:08:47.285 INFO:teuthology.orchestra.run.smithi097.stderr:114850K .......... .......... .......... .......... .......... 68% 36.9M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:114900K .......... .......... .......... .......... .......... 68% 25.6M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:114950K .......... .......... .......... .......... .......... 68% 21.7M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:115000K .......... .......... .......... .......... .......... 68% 47.3M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:115050K .......... .......... .......... .......... .......... 68% 23.5M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:115100K .......... .......... .......... .......... .......... 68% 34.6M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:115150K .......... .......... .......... .......... .......... 68% 22.9M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:115200K .......... .......... .......... .......... .......... 68% 55.6M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:115250K .......... .......... .......... .......... .......... 68% 25.6M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:115300K .......... .......... .......... .......... .......... 68% 23.9M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:115350K .......... .......... .......... .......... .......... 68% 20.1M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:115400K .......... .......... .......... .......... .......... 68% 41.4M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:115450K .......... .......... .......... .......... .......... 68% 54.6M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:115500K .......... .......... .......... .......... .......... 68% 27.2M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:115550K .......... .......... .......... .......... .......... 68% 28.5M 2s 2024-03-29T04:08:47.286 INFO:teuthology.orchestra.run.smithi097.stderr:115600K .......... .......... .......... .......... .......... 68% 23.9M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:115650K .......... .......... .......... .......... .......... 68% 22.3M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:115700K .......... .......... .......... .......... .......... 68% 30.4M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:115750K .......... .......... .......... .......... .......... 68% 26.3M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:115800K .......... .......... .......... .......... .......... 68% 48.0M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:115850K .......... .......... .......... .......... .......... 68% 28.3M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:115900K .......... .......... .......... .......... .......... 68% 24.3M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:115950K .......... .......... .......... .......... .......... 68% 54.3M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:116000K .......... .......... .......... .......... .......... 68% 30.5M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:116050K .......... .......... .......... .......... .......... 68% 25.2M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:116100K .......... .......... .......... .......... .......... 68% 24.0M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:116150K .......... .......... .......... .......... .......... 68% 52.9M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:116200K .......... .......... .......... .......... .......... 68% 22.0M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:116250K .......... .......... .......... .......... .......... 68% 39.0M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:116300K .......... .......... .......... .......... .......... 69% 24.1M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:116350K .......... .......... .......... .......... .......... 69% 25.2M 2s 2024-03-29T04:08:47.287 INFO:teuthology.orchestra.run.smithi097.stderr:116400K .......... .......... .......... .......... .......... 69% 53.2M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:116450K .......... .......... .......... .......... .......... 69% 24.0M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:116500K .......... .......... .......... .......... .......... 69% 33.7M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:116550K .......... .......... .......... .......... .......... 69% 24.2M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:116600K .......... .......... .......... .......... .......... 69% 24.5M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:116650K .......... .......... .......... .......... .......... 69% 52.0M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:116700K .......... .......... .......... .......... .......... 69% 23.2M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:116750K .......... .......... .......... .......... .......... 69% 34.4M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:116800K .......... .......... .......... .......... .......... 69% 21.8M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:116850K .......... .......... .......... .......... .......... 69% 44.5M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:116900K .......... .......... .......... .......... .......... 69% 23.9M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:116950K .......... .......... .......... .......... .......... 69% 23.9M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:117000K .......... .......... .......... .......... .......... 69% 51.1M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:117050K .......... .......... .......... .......... .......... 69% 24.6M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:117100K .......... .......... .......... .......... .......... 69% 24.7M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:117150K .......... .......... .......... .......... .......... 69% 32.5M 2s 2024-03-29T04:08:47.288 INFO:teuthology.orchestra.run.smithi097.stderr:117200K .......... .......... .......... .......... .......... 69% 30.8M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117250K .......... .......... .......... .......... .......... 69% 39.0M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117300K .......... .......... .......... .......... .......... 69% 25.4M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117350K .......... .......... .......... .......... .......... 69% 21.5M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117400K .......... .......... .......... .......... .......... 69% 49.5M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117450K .......... .......... .......... .......... .......... 69% 22.7M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117500K .......... .......... .......... .......... .......... 69% 38.9M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117550K .......... .......... .......... .......... .......... 69% 25.5M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117600K .......... .......... .......... .......... .......... 69% 33.8M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117650K .......... .......... .......... .......... .......... 69% 27.9M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117700K .......... .......... .......... .......... .......... 69% 21.9M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117750K .......... .......... .......... .......... .......... 69% 51.2M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117800K .......... .......... .......... .......... .......... 69% 23.7M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117850K .......... .......... .......... .......... .......... 69% 35.3M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117900K .......... .......... .......... .......... .......... 69% 24.7M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:117950K .......... .......... .......... .......... .......... 70% 24.7M 2s 2024-03-29T04:08:47.289 INFO:teuthology.orchestra.run.smithi097.stderr:118000K .......... .......... .......... .......... .......... 70% 46.8M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118050K .......... .......... .......... .......... .......... 70% 32.9M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118100K .......... .......... .......... .......... .......... 70% 17.2M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118150K .......... .......... .......... .......... .......... 70% 49.3M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118200K .......... .......... .......... .......... .......... 70% 24.2M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118250K .......... .......... .......... .......... .......... 70% 52.8M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118300K .......... .......... .......... .......... .......... 70% 29.4M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118350K .......... .......... .......... .......... .......... 70% 24.9M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118400K .......... .......... .......... .......... .......... 70% 26.1M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118450K .......... .......... .......... .......... .......... 70% 23.7M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118500K .......... .......... .......... .......... .......... 70% 63.5M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118550K .......... .......... .......... .......... .......... 70% 26.5M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118600K .......... .......... .......... .......... .......... 70% 29.4M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118650K .......... .......... .......... .......... .......... 70% 24.7M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118700K .......... .......... .......... .......... .......... 70% 24.2M 2s 2024-03-29T04:08:47.290 INFO:teuthology.orchestra.run.smithi097.stderr:118750K .......... .......... .......... .......... .......... 70% 38.5M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:118800K .......... .......... .......... .......... .......... 70% 33.1M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:118850K .......... .......... .......... .......... .......... 70% 29.8M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:118900K .......... .......... .......... .......... .......... 70% 25.0M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:118950K .......... .......... .......... .......... .......... 70% 24.2M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:119000K .......... .......... .......... .......... .......... 70% 53.3M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:119050K .......... .......... .......... .......... .......... 70% 24.2M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:119100K .......... .......... .......... .......... .......... 70% 23.8M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:119150K .......... .......... .......... .......... .......... 70% 35.1M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:119200K .......... .......... .......... .......... .......... 70% 23.6M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:119250K .......... .......... .......... .......... .......... 70% 53.5M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:119300K .......... .......... .......... .......... .......... 70% 22.5M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:119350K .......... .......... .......... .......... .......... 70% 38.6M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:119400K .......... .......... .......... .......... .......... 70% 25.0M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:119450K .......... .......... .......... .......... .......... 70% 24.8M 2s 2024-03-29T04:08:47.291 INFO:teuthology.orchestra.run.smithi097.stderr:119500K .......... .......... .......... .......... .......... 70% 46.3M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:119550K .......... .......... .......... .......... .......... 70% 25.3M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:119600K .......... .......... .......... .......... .......... 70% 16.6M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:119650K .......... .......... .......... .......... .......... 71% 82.2M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:119700K .......... .......... .......... .......... .......... 71% 21.8M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:119750K .......... .......... .......... .......... .......... 71% 89.1M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:119800K .......... .......... .......... .......... .......... 71% 24.1M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:119850K .......... .......... .......... .......... .......... 71% 21.0M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:119900K .......... .......... .......... .......... .......... 71% 36.7M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:119950K .......... .......... .......... .......... .......... 71% 27.5M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:120000K .......... .......... .......... .......... .......... 71% 24.0M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:120050K .......... .......... .......... .......... .......... 71% 49.1M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:120100K .......... .......... .......... .......... .......... 71% 29.5M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:120150K .......... .......... .......... .......... .......... 71% 28.5M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:120200K .......... .......... .......... .......... .......... 71% 24.7M 2s 2024-03-29T04:08:47.292 INFO:teuthology.orchestra.run.smithi097.stderr:120250K .......... .......... .......... .......... .......... 71% 24.4M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120300K .......... .......... .......... .......... .......... 71% 60.9M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120350K .......... .......... .......... .......... .......... 71% 29.1M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120400K .......... .......... .......... .......... .......... 71% 25.2M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120450K .......... .......... .......... .......... .......... 71% 24.9M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120500K .......... .......... .......... .......... .......... 71% 24.2M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120550K .......... .......... .......... .......... .......... 71% 49.0M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120600K .......... .......... .......... .......... .......... 71% 32.4M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120650K .......... .......... .......... .......... .......... 71% 25.9M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120700K .......... .......... .......... .......... .......... 71% 25.1M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120750K .......... .......... .......... .......... .......... 71% 24.4M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120800K .......... .......... .......... .......... .......... 71% 55.7M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120850K .......... .......... .......... .......... .......... 71% 27.5M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120900K .......... .......... .......... .......... .......... 71% 27.9M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:120950K .......... .......... .......... .......... .......... 71% 24.7M 2s 2024-03-29T04:08:47.293 INFO:teuthology.orchestra.run.smithi097.stderr:121000K .......... .......... .......... .......... .......... 71% 24.5M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121050K .......... .......... .......... .......... .......... 71% 55.9M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121100K .......... .......... .......... .......... .......... 71% 26.7M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121150K .......... .......... .......... .......... .......... 71% 28.3M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121200K .......... .......... .......... .......... .......... 71% 25.2M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121250K .......... .......... .......... .......... .......... 71% 24.5M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121300K .......... .......... .......... .......... .......... 71% 58.2M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121350K .......... .......... .......... .......... .......... 72% 24.4M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121400K .......... .......... .......... .......... .......... 72% 13.9M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121450K .......... .......... .......... .......... .......... 72% 127M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121500K .......... .......... .......... .......... .......... 72% 41.5M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121550K .......... .......... .......... .......... .......... 72% 32.9M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121600K .......... .......... .......... .......... .......... 72% 22.9M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121650K .......... .......... .......... .......... .......... 72% 23.6M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121700K .......... .......... .......... .......... .......... 72% 37.5M 2s 2024-03-29T04:08:47.294 INFO:teuthology.orchestra.run.smithi097.stderr:121750K .......... .......... .......... .......... .......... 72% 34.6M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:121800K .......... .......... .......... .......... .......... 72% 27.3M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:121850K .......... .......... .......... .......... .......... 72% 22.4M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:121900K .......... .......... .......... .......... .......... 72% 57.4M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:121950K .......... .......... .......... .......... .......... 72% 21.1M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:122000K .......... .......... .......... .......... .......... 72% 37.1M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:122050K .......... .......... .......... .......... .......... 72% 27.2M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:122100K .......... .......... .......... .......... .......... 72% 21.9M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:122150K .......... .......... .......... .......... .......... 72% 54.0M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:122200K .......... .......... .......... .......... .......... 72% 21.3M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:122250K .......... .......... .......... .......... .......... 72% 37.7M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:122300K .......... .......... .......... .......... .......... 72% 26.5M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:122350K .......... .......... .......... .......... .......... 72% 36.4M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:122400K .......... .......... .......... .......... .......... 72% 28.0M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:122450K .......... .......... .......... .......... .......... 72% 19.9M 2s 2024-03-29T04:08:47.295 INFO:teuthology.orchestra.run.smithi097.stderr:122500K .......... .......... .......... .......... .......... 72% 37.1M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:122550K .......... .......... .......... .......... .......... 72% 25.9M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:122600K .......... .......... .......... .......... .......... 72% 39.5M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:122650K .......... .......... .......... .......... .......... 72% 27.2M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:122700K .......... .......... .......... .......... .......... 72% 34.4M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:122750K .......... .......... .......... .......... .......... 72% 22.3M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:122800K .......... .......... .......... .......... .......... 72% 26.3M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:122850K .......... .......... .......... .......... .......... 72% 37.3M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:122900K .......... .......... .......... .......... .......... 72% 37.3M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:122950K .......... .......... .......... .......... .......... 72% 27.0M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:123000K .......... .......... .......... .......... .......... 72% 23.5M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:123050K .......... .......... .......... .......... .......... 73% 43.5M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:123100K .......... .......... .......... .......... .......... 73% 22.8M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:123150K .......... .......... .......... .......... .......... 73% 38.7M 2s 2024-03-29T04:08:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:123200K .......... .......... .......... .......... .......... 73% 24.8M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123250K .......... .......... .......... .......... .......... 73% 37.1M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123300K .......... .......... .......... .......... .......... 73% 18.1M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123350K .......... .......... .......... .......... .......... 73% 34.7M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123400K .......... .......... .......... .......... .......... 73% 38.9M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123450K .......... .......... .......... .......... .......... 73% 25.5M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123500K .......... .......... .......... .......... .......... 73% 38.3M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123550K .......... .......... .......... .......... .......... 73% 28.0M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123600K .......... .......... .......... .......... .......... 73% 29.6M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123650K .......... .......... .......... .......... .......... 73% 19.4M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123700K .......... .......... .......... .......... .......... 73% 31.5M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123750K .......... .......... .......... .......... .......... 73% 33.6M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123800K .......... .......... .......... .......... .......... 73% 33.5M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123850K .......... .......... .......... .......... .......... 73% 30.6M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123900K .......... .......... .......... .......... .......... 73% 32.1M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:123950K .......... .......... .......... .......... .......... 73% 19.7M 2s 2024-03-29T04:08:47.297 INFO:teuthology.orchestra.run.smithi097.stderr:124000K .......... .......... .......... .......... .......... 73% 37.1M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124050K .......... .......... .......... .......... .......... 73% 36.9M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124100K .......... .......... .......... .......... .......... 73% 17.7M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124150K .......... .......... .......... .......... .......... 73% 24.8M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124200K .......... .......... .......... .......... .......... 73% 105M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124250K .......... .......... .......... .......... .......... 73% 23.6M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124300K .......... .......... .......... .......... .......... 73% 24.5M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124350K .......... .......... .......... .......... .......... 73% 24.4M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124400K .......... .......... .......... .......... .......... 73% 24.0M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124450K .......... .......... .......... .......... .......... 73% 107M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124500K .......... .......... .......... .......... .......... 73% 23.7M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124550K .......... .......... .......... .......... .......... 73% 24.8M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124600K .......... .......... .......... .......... .......... 73% 23.9M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124650K .......... .......... .......... .......... .......... 73% 115M 2s 2024-03-29T04:08:47.298 INFO:teuthology.orchestra.run.smithi097.stderr:124700K .......... .......... .......... .......... .......... 74% 24.5M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:124750K .......... .......... .......... .......... .......... 74% 23.2M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:124800K .......... .......... .......... .......... .......... 74% 14.6M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:124850K .......... .......... .......... .......... .......... 74% 52.1M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:124900K .......... .......... .......... .......... .......... 74% 110M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:124950K .......... .......... .......... .......... .......... 74% 27.8M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:125000K .......... .......... .......... .......... .......... 74% 36.4M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:125050K .......... .......... .......... .......... .......... 74% 32.8M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:125100K .......... .......... .......... .......... .......... 74% 24.4M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:125150K .......... .......... .......... .......... .......... 74% 47.4M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:125200K .......... .......... .......... .......... .......... 74% 31.3M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:125250K .......... .......... .......... .......... .......... 74% 24.1M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:125300K .......... .......... .......... .......... .......... 74% 25.7M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:125350K .......... .......... .......... .......... .......... 74% 33.3M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:125400K .......... .......... .......... .......... .......... 74% 19.5M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:125450K .......... .......... .......... .......... .......... 74% 35.4M 2s 2024-03-29T04:08:47.299 INFO:teuthology.orchestra.run.smithi097.stderr:125500K .......... .......... .......... .......... .......... 74% 30.4M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:125550K .......... .......... .......... .......... .......... 74% 34.1M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:125600K .......... .......... .......... .......... .......... 74% 34.0M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:125650K .......... .......... .......... .......... .......... 74% 29.4M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:125700K .......... .......... .......... .......... .......... 74% 20.6M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:125750K .......... .......... .......... .......... .......... 74% 29.3M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:125800K .......... .......... .......... .......... .......... 74% 34.5M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:125850K .......... .......... .......... .......... .......... 74% 34.1M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:125900K .......... .......... .......... .......... .......... 74% 32.0M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:125950K .......... .......... .......... .......... .......... 74% 23.8M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:126000K .......... .......... .......... .......... .......... 74% 23.7M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:126050K .......... .......... .......... .......... .......... 74% 36.1M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:126100K .......... .......... .......... .......... .......... 74% 28.8M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:126150K .......... .......... .......... .......... .......... 74% 34.9M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:126200K .......... .......... .......... .......... .......... 74% 33.8M 2s 2024-03-29T04:08:47.300 INFO:teuthology.orchestra.run.smithi097.stderr:126250K .......... .......... .......... .......... .......... 74% 27.0M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126300K .......... .......... .......... .......... .......... 74% 26.4M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126350K .......... .......... .......... .......... .......... 74% 32.4M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126400K .......... .......... .......... .......... .......... 75% 34.1M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126450K .......... .......... .......... .......... .......... 75% 25.7M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126500K .......... .......... .......... .......... .......... 75% 23.1M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126550K .......... .......... .......... .......... .......... 75% 22.3M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126600K .......... .......... .......... .......... .......... 75% 23.9M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126650K .......... .......... .......... .......... .......... 75% 94.2M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126700K .......... .......... .......... .......... .......... 75% 25.6M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126750K .......... .......... .......... .......... .......... 75% 24.5M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126800K .......... .......... .......... .......... .......... 75% 23.0M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126850K .......... .......... .......... .......... .......... 75% 73.7M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126900K .......... .......... .......... .......... .......... 75% 16.0M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:126950K .......... .......... .......... .......... .......... 75% 18.1M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:127000K .......... .......... .......... .......... .......... 75% 42.0M 2s 2024-03-29T04:08:47.301 INFO:teuthology.orchestra.run.smithi097.stderr:127050K .......... .......... .......... .......... .......... 75% 57.2M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127100K .......... .......... .......... .......... .......... 75% 36.2M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127150K .......... .......... .......... .......... .......... 75% 31.5M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127200K .......... .......... .......... .......... .......... 75% 39.0M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127250K .......... .......... .......... .......... .......... 75% 25.2M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127300K .......... .......... .......... .......... .......... 75% 34.0M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127350K .......... .......... .......... .......... .......... 75% 25.8M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127400K .......... .......... .......... .......... .......... 75% 24.4M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127450K .......... .......... .......... .......... .......... 75% 29.3M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127500K .......... .......... .......... .......... .......... 75% 42.1M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127550K .......... .......... .......... .......... .......... 75% 15.8M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127600K .......... .......... .......... .......... .......... 75% 61.5M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127650K .......... .......... .......... .......... .......... 75% 24.9M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127700K .......... .......... .......... .......... .......... 75% 54.1M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127750K .......... .......... .......... .......... .......... 75% 42.0M 2s 2024-03-29T04:08:47.302 INFO:teuthology.orchestra.run.smithi097.stderr:127800K .......... .......... .......... .......... .......... 75% 19.6M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:127850K .......... .......... .......... .......... .......... 75% 36.4M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:127900K .......... .......... .......... .......... .......... 75% 39.9M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:127950K .......... .......... .......... .......... .......... 75% 24.4M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:128000K .......... .......... .......... .......... .......... 75% 25.1M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:128050K .......... .......... .......... .......... .......... 75% 24.5M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:128100K .......... .......... .......... .......... .......... 76% 33.9M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:128150K .......... .......... .......... .......... .......... 76% 38.4M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:128200K .......... .......... .......... .......... .......... 76% 27.9M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:128250K .......... .......... .......... .......... .......... 76% 26.7M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:128300K .......... .......... .......... .......... .......... 76% 23.5M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:128350K .......... .......... .......... .......... .......... 76% 30.1M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:128400K .......... .......... .......... .......... .......... 76% 53.0M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:128450K .......... .......... .......... .......... .......... 76% 22.9M 2s 2024-03-29T04:08:47.303 INFO:teuthology.orchestra.run.smithi097.stderr:128500K .......... .......... .......... .......... .......... 76% 27.4M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:128550K .......... .......... .......... .......... .......... 76% 33.0M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:128600K .......... .......... .......... .......... .......... 76% 21.1M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:128650K .......... .......... .......... .......... .......... 76% 35.6M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:128700K .......... .......... .......... .......... .......... 76% 37.9M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:128750K .......... .......... .......... .......... .......... 76% 35.9M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:128800K .......... .......... .......... .......... .......... 76% 24.9M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:128850K .......... .......... .......... .......... .......... 76% 21.2M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:128900K .......... .......... .......... .......... .......... 76% 28.6M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:128950K .......... .......... .......... .......... .......... 76% 48.2M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:129000K .......... .......... .......... .......... .......... 76% 29.4M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:129050K .......... .......... .......... .......... .......... 76% 28.4M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:129100K .......... .......... .......... .......... .......... 76% 24.8M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:129150K .......... .......... .......... .......... .......... 76% 25.0M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:129200K .......... .......... .......... .......... .......... 76% 46.2M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:129250K .......... .......... .......... .......... .......... 76% 25.1M 2s 2024-03-29T04:08:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:129300K .......... .......... .......... .......... .......... 76% 32.5M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:129350K .......... .......... .......... .......... .......... 76% 24.3M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:129400K .......... .......... .......... .......... .......... 76% 24.0M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:129450K .......... .......... .......... .......... .......... 76% 52.8M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:129500K .......... .......... .......... .......... .......... 76% 24.3M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:129550K .......... .......... .......... .......... .......... 76% 36.7M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:129600K .......... .......... .......... .......... .......... 76% 23.0M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:129650K .......... .......... .......... .......... .......... 76% 29.0M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:129700K .......... .......... .......... .......... .......... 76% 42.9M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:129750K .......... .......... .......... .......... .......... 77% 17.8M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:129800K .......... .......... .......... .......... .......... 77% 31.7M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:129850K .......... .......... .......... .......... .......... 77% 42.2M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:129900K .......... .......... .......... .......... .......... 77% 32.2M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:129950K .......... .......... .......... .......... .......... 77% 29.8M 2s 2024-03-29T04:08:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:130000K .......... .......... .......... .......... .......... 77% 17.6M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130050K .......... .......... .......... .......... .......... 77% 39.4M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130100K .......... .......... .......... .......... .......... 77% 24.8M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130150K .......... .......... .......... .......... .......... 77% 39.5M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130200K .......... .......... .......... .......... .......... 77% 53.8M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130250K .......... .......... .......... .......... .......... 77% 22.8M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130300K .......... .......... .......... .......... .......... 77% 16.0M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130350K .......... .......... .......... .......... .......... 77% 55.7M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130400K .......... .......... .......... .......... .......... 77% 36.0M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130450K .......... .......... .......... .......... .......... 77% 52.6M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130500K .......... .......... .......... .......... .......... 77% 23.2M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130550K .......... .......... .......... .......... .......... 77% 27.6M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130600K .......... .......... .......... .......... .......... 77% 27.0M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130650K .......... .......... .......... .......... .......... 77% 25.8M 2s 2024-03-29T04:08:47.306 INFO:teuthology.orchestra.run.smithi097.stderr:130700K .......... .......... .......... .......... .......... 77% 64.9M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:130750K .......... .......... .......... .......... .......... 77% 23.0M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:130800K .......... .......... .......... .......... .......... 77% 30.3M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:130850K .......... .......... .......... .......... .......... 77% 27.6M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:130900K .......... .......... .......... .......... .......... 77% 25.4M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:130950K .......... .......... .......... .......... .......... 77% 43.9M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:131000K .......... .......... .......... .......... .......... 77% 25.8M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:131050K .......... .......... .......... .......... .......... 77% 23.4M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:131100K .......... .......... .......... .......... .......... 77% 30.1M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:131150K .......... .......... .......... .......... .......... 77% 22.8M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:131200K .......... .......... .......... .......... .......... 77% 17.6M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:131250K .......... .......... .......... .......... .......... 77% 60.2M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:131300K .......... .......... .......... .......... .......... 77% 49.3M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:131350K .......... .......... .......... .......... .......... 77% 30.4M 2s 2024-03-29T04:08:47.307 INFO:teuthology.orchestra.run.smithi097.stderr:131400K .......... .......... .......... .......... .......... 77% 28.9M 2s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:131450K .......... .......... .......... .......... .......... 78% 37.3M 2s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:131500K .......... .......... .......... .......... .......... 78% 26.2M 1s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:131550K .......... .......... .......... .......... .......... 78% 27.9M 1s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:131600K .......... .......... .......... .......... .......... 78% 32.2M 1s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:131650K .......... .......... .......... .......... .......... 78% 24.3M 1s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:131700K .......... .......... .......... .......... .......... 78% 24.8M 1s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:131750K .......... .......... .......... .......... .......... 78% 50.0M 1s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:131800K .......... .......... .......... .......... .......... 78% 24.0M 1s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:131850K .......... .......... .......... .......... .......... 78% 35.5M 1s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:131900K .......... .......... .......... .......... .......... 78% 20.8M 1s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:131950K .......... .......... .......... .......... .......... 78% 27.5M 1s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:132000K .......... .......... .......... .......... .......... 78% 33.7M 1s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:132050K .......... .......... .......... .......... .......... 78% 20.6M 1s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:132100K .......... .......... .......... .......... .......... 78% 41.3M 1s 2024-03-29T04:08:47.308 INFO:teuthology.orchestra.run.smithi097.stderr:132150K .......... .......... .......... .......... .......... 78% 37.3M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132200K .......... .......... .......... .......... .......... 78% 25.4M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132250K .......... .......... .......... .......... .......... 78% 43.2M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132300K .......... .......... .......... .......... .......... 78% 23.3M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132350K .......... .......... .......... .......... .......... 78% 39.6M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132400K .......... .......... .......... .......... .......... 78% 31.7M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132450K .......... .......... .......... .......... .......... 78% 20.5M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132500K .......... .......... .......... .......... .......... 78% 44.3M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132550K .......... .......... .......... .......... .......... 78% 24.7M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132600K .......... .......... .......... .......... .......... 78% 24.0M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132650K .......... .......... .......... .......... .......... 78% 37.0M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132700K .......... .......... .......... .......... .......... 78% 47.1M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132750K .......... .......... .......... .......... .......... 78% 24.9M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132800K .......... .......... .......... .......... .......... 78% 26.2M 1s 2024-03-29T04:08:47.309 INFO:teuthology.orchestra.run.smithi097.stderr:132850K .......... .......... .......... .......... .......... 78% 18.7M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:132900K .......... .......... .......... .......... .......... 78% 18.4M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:132950K .......... .......... .......... .......... .......... 78% 36.0M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:133000K .......... .......... .......... .......... .......... 78% 40.7M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:133050K .......... .......... .......... .......... .......... 78% 70.0M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:133100K .......... .......... .......... .......... .......... 78% 30.6M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:133150K .......... .......... .......... .......... .......... 79% 22.4M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:133200K .......... .......... .......... .......... .......... 79% 33.0M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:133250K .......... .......... .......... .......... .......... 79% 31.8M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:133300K .......... .......... .......... .......... .......... 79% 27.4M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:133350K .......... .......... .......... .......... .......... 79% 26.5M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:133400K .......... .......... .......... .......... .......... 79% 47.1M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:133450K .......... .......... .......... .......... .......... 79% 26.5M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:133500K .......... .......... .......... .......... .......... 79% 29.6M 1s 2024-03-29T04:08:47.310 INFO:teuthology.orchestra.run.smithi097.stderr:133550K .......... .......... .......... .......... .......... 79% 29.0M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:133600K .......... .......... .......... .......... .......... 79% 21.9M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:133650K .......... .......... .......... .......... .......... 79% 35.2M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:133700K .......... .......... .......... .......... .......... 79% 23.1M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:133750K .......... .......... .......... .......... .......... 79% 41.3M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:133800K .......... .......... .......... .......... .......... 79% 22.7M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:133850K .......... .......... .......... .......... .......... 79% 26.1M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:133900K .......... .......... .......... .......... .......... 79% 27.6M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:133950K .......... .......... .......... .......... .......... 79% 44.9M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:134000K .......... .......... .......... .......... .......... 79% 30.7M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:134050K .......... .......... .......... .......... .......... 79% 25.7M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:134100K .......... .......... .......... .......... .......... 79% 24.4M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:134150K .......... .......... .......... .......... .......... 79% 23.5M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:134200K .......... .......... .......... .......... .......... 79% 63.7M 1s 2024-03-29T04:08:47.311 INFO:teuthology.orchestra.run.smithi097.stderr:134250K .......... .......... .......... .......... .......... 79% 26.9M 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134300K .......... .......... .......... .......... .......... 79% 30.9M 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134350K .......... .......... .......... .......... .......... 79% 312K 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134400K .......... .......... .......... .......... .......... 79% 557M 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134450K .......... .......... .......... .......... .......... 79% 657M 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134500K .......... .......... .......... .......... .......... 79% 664M 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134550K .......... .......... .......... .......... .......... 79% 573M 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134600K .......... .......... .......... .......... .......... 79% 625M 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134650K .......... .......... .......... .......... .......... 79% 639M 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134700K .......... .......... .......... .......... .......... 79% 663M 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134750K .......... .......... .......... .......... .......... 79% 618M 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134800K .......... .......... .......... .......... .......... 79% 663M 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134850K .......... .......... .......... .......... .......... 80% 683M 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134900K .......... .......... .......... .......... .......... 80% 696M 1s 2024-03-29T04:08:47.312 INFO:teuthology.orchestra.run.smithi097.stderr:134950K .......... .......... .......... .......... .......... 80% 631M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135000K .......... .......... .......... .......... .......... 80% 732M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135050K .......... .......... .......... .......... .......... 80% 719M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135100K .......... .......... .......... .......... .......... 80% 717M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135150K .......... .......... .......... .......... .......... 80% 623M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135200K .......... .......... .......... .......... .......... 80% 686M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135250K .......... .......... .......... .......... .......... 80% 726M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135300K .......... .......... .......... .......... .......... 80% 716M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135350K .......... .......... .......... .......... .......... 80% 605M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135400K .......... .......... .......... .......... .......... 80% 724M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135450K .......... .......... .......... .......... .......... 80% 750M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135500K .......... .......... .......... .......... .......... 80% 751M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135550K .......... .......... .......... .......... .......... 80% 604M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135600K .......... .......... .......... .......... .......... 80% 713M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135650K .......... .......... .......... .......... .......... 80% 734M 1s 2024-03-29T04:08:47.313 INFO:teuthology.orchestra.run.smithi097.stderr:135700K .......... .......... .......... .......... .......... 80% 763M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:135750K .......... .......... .......... .......... .......... 80% 606M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:135800K .......... .......... .......... .......... .......... 80% 675M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:135850K .......... .......... .......... .......... .......... 80% 692M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:135900K .......... .......... .......... .......... .......... 80% 721M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:135950K .......... .......... .......... .......... .......... 80% 636M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:136000K .......... .......... .......... .......... .......... 80% 746M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:136050K .......... .......... .......... .......... .......... 80% 747M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:136100K .......... .......... .......... .......... .......... 80% 706M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:136150K .......... .......... .......... .......... .......... 80% 628M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:136200K .......... .......... .......... .......... .......... 80% 721M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:136250K .......... .......... .......... .......... .......... 80% 710M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:136300K .......... .......... .......... .......... .......... 80% 758M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:136350K .......... .......... .......... .......... .......... 80% 625M 1s 2024-03-29T04:08:47.314 INFO:teuthology.orchestra.run.smithi097.stderr:136400K .......... .......... .......... .......... .......... 80% 742M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:136450K .......... .......... .......... .......... .......... 80% 703M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:136500K .......... .......... .......... .......... .......... 81% 750M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:136550K .......... .......... .......... .......... .......... 81% 649M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:136600K .......... .......... .......... .......... .......... 81% 676M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:136650K .......... .......... .......... .......... .......... 81% 723M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:136700K .......... .......... .......... .......... .......... 81% 720M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:136750K .......... .......... .......... .......... .......... 81% 627M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:136800K .......... .......... .......... .......... .......... 81% 723M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:136850K .......... .......... .......... .......... .......... 81% 711M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:136900K .......... .......... .......... .......... .......... 81% 754M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:136950K .......... .......... .......... .......... .......... 81% 580M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:137000K .......... .......... .......... .......... .......... 81% 738M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:137050K .......... .......... .......... .......... .......... 81% 731M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:137100K .......... .......... .......... .......... .......... 81% 703M 1s 2024-03-29T04:08:47.315 INFO:teuthology.orchestra.run.smithi097.stderr:137150K .......... .......... .......... .......... .......... 81% 633M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137200K .......... .......... .......... .......... .......... 81% 715M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137250K .......... .......... .......... .......... .......... 81% 741M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137300K .......... .......... .......... .......... .......... 81% 735M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137350K .......... .......... .......... .......... .......... 81% 611M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137400K .......... .......... .......... .......... .......... 81% 716M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137450K .......... .......... .......... .......... .......... 81% 725M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137500K .......... .......... .......... .......... .......... 81% 748M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137550K .......... .......... .......... .......... .......... 81% 632M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137600K .......... .......... .......... .......... .......... 81% 706M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137650K .......... .......... .......... .......... .......... 81% 707M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137700K .......... .......... .......... .......... .......... 81% 746M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137750K .......... .......... .......... .......... .......... 81% 635M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137800K .......... .......... .......... .......... .......... 81% 726M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137850K .......... .......... .......... .......... .......... 81% 714M 1s 2024-03-29T04:08:47.316 INFO:teuthology.orchestra.run.smithi097.stderr:137900K .......... .......... .......... .......... .......... 81% 750M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:137950K .......... .......... .......... .......... .......... 81% 635M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138000K .......... .......... .......... .......... .......... 81% 740M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138050K .......... .......... .......... .......... .......... 81% 739M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138100K .......... .......... .......... .......... .......... 81% 722M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138150K .......... .......... .......... .......... .......... 81% 636M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138200K .......... .......... .......... .......... .......... 82% 723M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138250K .......... .......... .......... .......... .......... 82% 746M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138300K .......... .......... .......... .......... .......... 82% 736M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138350K .......... .......... .......... .......... .......... 82% 640M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138400K .......... .......... .......... .......... .......... 82% 719M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138450K .......... .......... .......... .......... .......... 82% 1.61M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138500K .......... .......... .......... .......... .......... 82% 106M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138550K .......... .......... .......... .......... .......... 82% 115M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138600K .......... .......... .......... .......... .......... 82% 116M 1s 2024-03-29T04:08:47.317 INFO:teuthology.orchestra.run.smithi097.stderr:138650K .......... .......... .......... .......... .......... 82% 83.3M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:138700K .......... .......... .......... .......... .......... 82% 109M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:138750K .......... .......... .......... .......... .......... 82% 40.1M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:138800K .......... .......... .......... .......... .......... 82% 60.0M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:138850K .......... .......... .......... .......... .......... 82% 57.1M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:138900K .......... .......... .......... .......... .......... 82% 31.8M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:138950K .......... .......... .......... .......... .......... 82% 26.8M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:139000K .......... .......... .......... .......... .......... 82% 21.1M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:139050K .......... .......... .......... .......... .......... 82% 77.4M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:139100K .......... .......... .......... .......... .......... 82% 24.1M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:139150K .......... .......... .......... .......... .......... 82% 33.9M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:139200K .......... .......... .......... .......... .......... 82% 24.7M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:139250K .......... .......... .......... .......... .......... 82% 24.7M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:139300K .......... .......... .......... .......... .......... 82% 49.6M 1s 2024-03-29T04:08:47.318 INFO:teuthology.orchestra.run.smithi097.stderr:139350K .......... .......... .......... .......... .......... 82% 30.3M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:139400K .......... .......... .......... .......... .......... 82% 27.2M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:139450K .......... .......... .......... .......... .......... 82% 22.0M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:139500K .......... .......... .......... .......... .......... 82% 27.8M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:139550K .......... .......... .......... .......... .......... 82% 50.0M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:139600K .......... .......... .......... .......... .......... 82% 24.5M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:139650K .......... .......... .......... .......... .......... 82% 32.7M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:139700K .......... .......... .......... .......... .......... 82% 25.3M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:139750K .......... .......... .......... .......... .......... 82% 25.0M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:139800K .......... .......... .......... .......... .......... 82% 50.8M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:139850K .......... .......... .......... .......... .......... 82% 24.7M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:139900K .......... .......... .......... .......... .......... 83% 33.5M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:139950K .......... .......... .......... .......... .......... 83% 23.9M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:140000K .......... .......... .......... .......... .......... 83% 25.6M 1s 2024-03-29T04:08:47.319 INFO:teuthology.orchestra.run.smithi097.stderr:140050K .......... .......... .......... .......... .......... 83% 49.8M 1s 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:140100K .......... .......... .......... .......... .......... 83% 23.0M 1s 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:140150K .......... .......... .......... .......... .......... 83% 37.2M 1s 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:140200K .......... .......... .......... .......... .......... 83% 24.5M 1s 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:140250K .......... .......... .......... .......... .......... 83% 25.0M 1s 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:140300K .......... .......... .......... .......... .......... 83% 48.8M 1s 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:140350K .......... .......... .......... .......... .......... 83% 21.9M 1s 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:140400K .......... .......... .......... .......... .......... 83% 34.4M 1s 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:140450K .......... .......... .......... .......... .......... 83% 24.8M 1s 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:140500K .......... .......... .......... .......... .......... 83% 11.7M 1s 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:140550K .......... .......... .......... .......... .......... 83% 118M 1s 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:140600K .......... .......... .......... .......... .......... 83% 54.4M 1s 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:140650K .......... .......... .......... .......... .......... 83% 29.9M 1s 2024-03-29T04:08:47.320 INFO:teuthology.orchestra.run.smithi097.stderr:140700K .......... .......... .......... .......... .......... 83% 25.3M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:140750K .......... .......... .......... .......... .......... 83% 24.5M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:140800K .......... .......... .......... .......... .......... 83% 112M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:140850K .......... .......... .......... .......... .......... 83% 22.4M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:140900K .......... .......... .......... .......... .......... 83% 25.9M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:140950K .......... .......... .......... .......... .......... 83% 24.5M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:141000K .......... .......... .......... .......... .......... 83% 24.8M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:141050K .......... .......... .......... .......... .......... 83% 113M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:141100K .......... .......... .......... .......... .......... 83% 24.6M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:141150K .......... .......... .......... .......... .......... 83% 24.5M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:141200K .......... .......... .......... .......... .......... 83% 24.6M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:141250K .......... .......... .......... .......... .......... 83% 25.7M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:141300K .......... .......... .......... .......... .......... 83% 112M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:141350K .......... .......... .......... .......... .......... 83% 23.0M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:141400K .......... .......... .......... .......... .......... 83% 15.3M 1s 2024-03-29T04:08:47.321 INFO:teuthology.orchestra.run.smithi097.stderr:141450K .......... .......... .......... .......... .......... 83% 35.7M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:141500K .......... .......... .......... .......... .......... 83% 28.0M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:141550K .......... .......... .......... .......... .......... 84% 96.1M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:141600K .......... .......... .......... .......... .......... 84% 29.9M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:141650K .......... .......... .......... .......... .......... 84% 25.5M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:141700K .......... .......... .......... .......... .......... 84% 25.5M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:141750K .......... .......... .......... .......... .......... 84% 24.4M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:141800K .......... .......... .......... .......... .......... 84% 75.4M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:141850K .......... .......... .......... .......... .......... 84% 16.4M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:141900K .......... .......... .......... .......... .......... 84% 16.0M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:141950K .......... .......... .......... .......... .......... 84% 40.3M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:142000K .......... .......... .......... .......... .......... 84% 48.2M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:142050K .......... .......... .......... .......... .......... 84% 62.5M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:142100K .......... .......... .......... .......... .......... 84% 39.0M 1s 2024-03-29T04:08:47.322 INFO:teuthology.orchestra.run.smithi097.stderr:142150K .......... .......... .......... .......... .......... 84% 31.6M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142200K .......... .......... .......... .......... .......... 84% 22.9M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142250K .......... .......... .......... .......... .......... 84% 21.7M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142300K .......... .......... .......... .......... .......... 84% 28.1M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142350K .......... .......... .......... .......... .......... 84% 73.1M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142400K .......... .......... .......... .......... .......... 84% 29.6M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142450K .......... .......... .......... .......... .......... 84% 24.1M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142500K .......... .......... .......... .......... .......... 84% 25.3M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142550K .......... .......... .......... .......... .......... 84% 65.7M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142600K .......... .......... .......... .......... .......... 84% 24.3M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142650K .......... .......... .......... .......... .......... 84% 29.5M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142700K .......... .......... .......... .......... .......... 84% 22.1M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142750K .......... .......... .......... .......... .......... 84% 12.9M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142800K .......... .......... .......... .......... .......... 84% 34.5M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142850K .......... .......... .......... .......... .......... 84% 17.3M 1s 2024-03-29T04:08:47.323 INFO:teuthology.orchestra.run.smithi097.stderr:142900K .......... .......... .......... .......... .......... 84% 32.9M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:142950K .......... .......... .......... .......... .......... 84% 28.7M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143000K .......... .......... .......... .......... .......... 84% 27.5M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143050K .......... .......... .......... .......... .......... 84% 58.6M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143100K .......... .......... .......... .......... .......... 84% 38.4M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143150K .......... .......... .......... .......... .......... 84% 36.2M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143200K .......... .......... .......... .......... .......... 84% 94.7M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143250K .......... .......... .......... .......... .......... 85% 27.7M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143300K .......... .......... .......... .......... .......... 85% 79.2M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143350K .......... .......... .......... .......... .......... 85% 26.7M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143400K .......... .......... .......... .......... .......... 85% 25.0M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143450K .......... .......... .......... .......... .......... 85% 25.6M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143500K .......... .......... .......... .......... .......... 85% 21.5M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143550K .......... .......... .......... .......... .......... 85% 106M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143600K .......... .......... .......... .......... .......... 85% 26.9M 1s 2024-03-29T04:08:47.324 INFO:teuthology.orchestra.run.smithi097.stderr:143650K .......... .......... .......... .......... .......... 85% 24.1M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:143700K .......... .......... .......... .......... .......... 85% 23.3M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:143750K .......... .......... .......... .......... .......... 85% 55.0M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:143800K .......... .......... .......... .......... .......... 85% 31.0M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:143850K .......... .......... .......... .......... .......... 85% 26.0M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:143900K .......... .......... .......... .......... .......... 85% 24.6M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:143950K .......... .......... .......... .......... .......... 85% 25.1M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:144000K .......... .......... .......... .......... .......... 85% 39.5M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:144050K .......... .......... .......... .......... .......... 85% 28.6M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:144100K .......... .......... .......... .......... .......... 85% 33.3M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:144150K .......... .......... .......... .......... .......... 85% 22.0M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:144200K .......... .......... .......... .......... .......... 85% 27.7M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:144250K .......... .......... .......... .......... .......... 85% 54.2M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:144300K .......... .......... .......... .......... .......... 85% 25.8M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:144350K .......... .......... .......... .......... .......... 85% 31.1M 1s 2024-03-29T04:08:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:144400K .......... .......... .......... .......... .......... 85% 24.6M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:144450K .......... .......... .......... .......... .......... 85% 22.7M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:144500K .......... .......... .......... .......... .......... 85% 44.9M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:144550K .......... .......... .......... .......... .......... 85% 25.9M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:144600K .......... .......... .......... .......... .......... 85% 36.7M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:144650K .......... .......... .......... .......... .......... 85% 24.6M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:144700K .......... .......... .......... .......... .......... 85% 24.0M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:144750K .......... .......... .......... .......... .......... 85% 57.8M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:144800K .......... .......... .......... .......... .......... 85% 25.5M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:144850K .......... .......... .......... .......... .......... 85% 26.1M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:144900K .......... .......... .......... .......... .......... 85% 29.0M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:144950K .......... .......... .......... .......... .......... 86% 24.2M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:145000K .......... .......... .......... .......... .......... 86% 49.5M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:145050K .......... .......... .......... .......... .......... 86% 23.6M 1s 2024-03-29T04:08:47.326 INFO:teuthology.orchestra.run.smithi097.stderr:145100K .......... .......... .......... .......... .......... 86% 37.2M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145150K .......... .......... .......... .......... .......... 86% 23.7M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145200K .......... .......... .......... .......... .......... 86% 25.3M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145250K .......... .......... .......... .......... .......... 86% 42.3M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145300K .......... .......... .......... .......... .......... 86% 16.8M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145350K .......... .......... .......... .......... .......... 86% 92.1M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145400K .......... .......... .......... .......... .......... 86% 26.3M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145450K .......... .......... .......... .......... .......... 86% 28.4M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145500K .......... .......... .......... .......... .......... 86% 40.6M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145550K .......... .......... .......... .......... .......... 86% 27.7M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145600K .......... .......... .......... .......... .......... 86% 28.8M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145650K .......... .......... .......... .......... .......... 86% 23.9M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145700K .......... .......... .......... .......... .......... 86% 35.7M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145750K .......... .......... .......... .......... .......... 86% 21.3M 1s 2024-03-29T04:08:47.327 INFO:teuthology.orchestra.run.smithi097.stderr:145800K .......... .......... .......... .......... .......... 86% 45.9M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:145850K .......... .......... .......... .......... .......... 86% 24.7M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:145900K .......... .......... .......... .......... .......... 86% 26.7M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:145950K .......... .......... .......... .......... .......... 86% 22.3M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:146000K .......... .......... .......... .......... .......... 86% 44.5M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:146050K .......... .......... .......... .......... .......... 86% 19.4M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:146100K .......... .......... .......... .......... .......... 86% 35.5M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:146150K .......... .......... .......... .......... .......... 86% 35.3M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:146200K .......... .......... .......... .......... .......... 86% 48.3M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:146250K .......... .......... .......... .......... .......... 86% 14.5M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:146300K .......... .......... .......... .......... .......... 86% 44.5M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:146350K .......... .......... .......... .......... .......... 86% 36.9M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:146400K .......... .......... .......... .......... .......... 86% 37.9M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:146450K .......... .......... .......... .......... .......... 86% 26.4M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:146500K .......... .......... .......... .......... .......... 86% 28.9M 1s 2024-03-29T04:08:47.328 INFO:teuthology.orchestra.run.smithi097.stderr:146550K .......... .......... .......... .......... .......... 86% 38.1M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:146600K .......... .......... .......... .......... .......... 86% 23.9M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:146650K .......... .......... .......... .......... .......... 87% 30.3M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:146700K .......... .......... .......... .......... .......... 87% 25.2M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:146750K .......... .......... .......... .......... .......... 87% 26.6M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:146800K .......... .......... .......... .......... .......... 87% 39.6M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:146850K .......... .......... .......... .......... .......... 87% 24.8M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:146900K .......... .......... .......... .......... .......... 87% 40.0M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:146950K .......... .......... .......... .......... .......... 87% 25.8M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:147000K .......... .......... .......... .......... .......... 87% 22.3M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:147050K .......... .......... .......... .......... .......... 87% 46.6M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:147100K .......... .......... .......... .......... .......... 87% 24.2M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:147150K .......... .......... .......... .......... .......... 87% 34.4M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:147200K .......... .......... .......... .......... .......... 87% 24.6M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:147250K .......... .......... .......... .......... .......... 87% 24.1M 1s 2024-03-29T04:08:47.329 INFO:teuthology.orchestra.run.smithi097.stderr:147300K .......... .......... .......... .......... .......... 87% 47.0M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:147350K .......... .......... .......... .......... .......... 87% 26.5M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:147400K .......... .......... .......... .......... .......... 87% 24.2M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:147450K .......... .......... .......... .......... .......... 87% 36.1M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:147500K .......... .......... .......... .......... .......... 87% 24.7M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:147550K .......... .......... .......... .......... .......... 87% 46.4M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:147600K .......... .......... .......... .......... .......... 87% 24.6M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:147650K .......... .......... .......... .......... .......... 87% 32.4M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:147700K .......... .......... .......... .......... .......... 87% 26.2M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:147750K .......... .......... .......... .......... .......... 87% 24.8M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:147800K .......... .......... .......... .......... .......... 87% 46.4M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:147850K .......... .......... .......... .......... .......... 87% 34.3M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:147900K .......... .......... .......... .......... .......... 87% 26.1M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:147950K .......... .......... .......... .......... .......... 87% 23.9M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:148000K .......... .......... .......... .......... .......... 87% 26.8M 1s 2024-03-29T04:08:47.330 INFO:teuthology.orchestra.run.smithi097.stderr:148050K .......... .......... .......... .......... .......... 87% 39.4M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148100K .......... .......... .......... .......... .......... 87% 32.3M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148150K .......... .......... .......... .......... .......... 87% 27.2M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148200K .......... .......... .......... .......... .......... 87% 23.0M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148250K .......... .......... .......... .......... .......... 87% 27.6M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148300K .......... .......... .......... .......... .......... 88% 52.1M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148350K .......... .......... .......... .......... .......... 88% 22.0M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148400K .......... .......... .......... .......... .......... 88% 40.2M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148450K .......... .......... .......... .......... .......... 88% 23.5M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148500K .......... .......... .......... .......... .......... 88% 24.0M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148550K .......... .......... .......... .......... .......... 88% 43.7M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148600K .......... .......... .......... .......... .......... 88% 34.3M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148650K .......... .......... .......... .......... .......... 88% 24.5M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148700K .......... .......... .......... .......... .......... 88% 23.7M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148750K .......... .......... .......... .......... .......... 88% 23.1M 1s 2024-03-29T04:08:47.331 INFO:teuthology.orchestra.run.smithi097.stderr:148800K .......... .......... .......... .......... .......... 88% 34.9M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:148850K .......... .......... .......... .......... .......... 88% 25.4M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:148900K .......... .......... .......... .......... .......... 88% 42.6M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:148950K .......... .......... .......... .......... .......... 88% 32.4M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:149000K .......... .......... .......... .......... .......... 88% 24.2M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:149050K .......... .......... .......... .......... .......... 88% 46.0M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:149100K .......... .......... .......... .......... .......... 88% 27.6M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:149150K .......... .......... .......... .......... .......... 88% 31.3M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:149200K .......... .......... .......... .......... .......... 88% 24.2M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:149250K .......... .......... .......... .......... .......... 88% 25.0M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:149300K .......... .......... .......... .......... .......... 88% 46.2M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:149350K .......... .......... .......... .......... .......... 88% 25.7M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:149400K .......... .......... .......... .......... .......... 88% 23.2M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:149450K .......... .......... .......... .......... .......... 88% 31.4M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:149500K .......... .......... .......... .......... .......... 88% 48.5M 1s 2024-03-29T04:08:47.332 INFO:teuthology.orchestra.run.smithi097.stderr:149550K .......... .......... .......... .......... .......... 88% 28.5M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:149600K .......... .......... .......... .......... .......... 88% 24.1M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:149650K .......... .......... .......... .......... .......... 88% 34.6M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:149700K .......... .......... .......... .......... .......... 88% 10.3M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:149750K .......... .......... .......... .......... .......... 88% 200M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:149800K .......... .......... .......... .......... .......... 88% 179M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:149850K .......... .......... .......... .......... .......... 88% 36.9M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:149900K .......... .......... .......... .......... .......... 88% 24.0M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:149950K .......... .......... .......... .......... .......... 88% 28.1M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:150000K .......... .......... .......... .......... .......... 89% 37.9M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:150050K .......... .......... .......... .......... .......... 89% 29.1M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:150100K .......... .......... .......... .......... .......... 89% 24.3M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:150150K .......... .......... .......... .......... .......... 89% 27.6M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:150200K .......... .......... .......... .......... .......... 89% 8.40M 1s 2024-03-29T04:08:47.333 INFO:teuthology.orchestra.run.smithi097.stderr:150250K .......... .......... .......... .......... .......... 89% 15.3M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150300K .......... .......... .......... .......... .......... 89% 13.1M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150350K .......... .......... .......... .......... .......... 89% 19.3M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150400K .......... .......... .......... .......... .......... 89% 18.0M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150450K .......... .......... .......... .......... .......... 89% 12.2M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150500K .......... .......... .......... .......... .......... 89% 107M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150550K .......... .......... .......... .......... .......... 89% 21.7M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150600K .......... .......... .......... .......... .......... 89% 50.1M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150650K .......... .......... .......... .......... .......... 89% 39.7M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150700K .......... .......... .......... .......... .......... 89% 90.9M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150750K .......... .......... .......... .......... .......... 89% 104M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150800K .......... .......... .......... .......... .......... 89% 101M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150850K .......... .......... .......... .......... .......... 89% 108M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150900K .......... .......... .......... .......... .......... 89% 86.0M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:150950K .......... .......... .......... .......... .......... 89% 92.8M 1s 2024-03-29T04:08:47.334 INFO:teuthology.orchestra.run.smithi097.stderr:151000K .......... .......... .......... .......... .......... 89% 59.5M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151050K .......... .......... .......... .......... .......... 89% 51.9M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151100K .......... .......... .......... .......... .......... 89% 94.6M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151150K .......... .......... .......... .......... .......... 89% 36.1M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151200K .......... .......... .......... .......... .......... 89% 25.0M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151250K .......... .......... .......... .......... .......... 89% 46.4M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151300K .......... .......... .......... .......... .......... 89% 24.6M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151350K .......... .......... .......... .......... .......... 89% 35.6M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151400K .......... .......... .......... .......... .......... 89% 28.6M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151450K .......... .......... .......... .......... .......... 89% 15.7M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151500K .......... .......... .......... .......... .......... 89% 93.1M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151550K .......... .......... .......... .......... .......... 89% 29.7M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151600K .......... .......... .......... .......... .......... 89% 29.9M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151650K .......... .......... .......... .......... .......... 89% 27.8M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151700K .......... .......... .......... .......... .......... 90% 22.1M 1s 2024-03-29T04:08:47.335 INFO:teuthology.orchestra.run.smithi097.stderr:151750K .......... .......... .......... .......... .......... 90% 54.0M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:151800K .......... .......... .......... .......... .......... 90% 26.5M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:151850K .......... .......... .......... .......... .......... 90% 34.5M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:151900K .......... .......... .......... .......... .......... 90% 24.7M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:151950K .......... .......... .......... .......... .......... 90% 24.8M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:152000K .......... .......... .......... .......... .......... 90% 54.9M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:152050K .......... .......... .......... .......... .......... 90% 20.2M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:152100K .......... .......... .......... .......... .......... 90% 33.2M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:152150K .......... .......... .......... .......... .......... 90% 22.7M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:152200K .......... .......... .......... .......... .......... 90% 21.2M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:152250K .......... .......... .......... .......... .......... 90% 24.1M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:152300K .......... .......... .......... .......... .......... 90% 25.2M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:152350K .......... .......... .......... .......... .......... 90% 22.8M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:152400K .......... .......... .......... .......... .......... 90% 9.18M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:152450K .......... .......... .......... .......... .......... 90% 98.5M 1s 2024-03-29T04:08:47.336 INFO:teuthology.orchestra.run.smithi097.stderr:152500K .......... .......... .......... .......... .......... 90% 38.4M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:152550K .......... .......... .......... .......... .......... 90% 11.0M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:152600K .......... .......... .......... .......... .......... 90% 42.5M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:152650K .......... .......... .......... .......... .......... 90% 57.8M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:152700K .......... .......... .......... .......... .......... 90% 97.5M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:152750K .......... .......... .......... .......... .......... 90% 36.2M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:152800K .......... .......... .......... .......... .......... 90% 21.1M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:152850K .......... .......... .......... .......... .......... 90% 29.1M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:152900K .......... .......... .......... .......... .......... 90% 68.5M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:152950K .......... .......... .......... .......... .......... 90% 17.8M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:153000K .......... .......... .......... .......... .......... 90% 18.3M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:153050K .......... .......... .......... .......... .......... 90% 19.8M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:153100K .......... .......... .......... .......... .......... 90% 21.2M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:153150K .......... .......... .......... .......... .......... 90% 48.6M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:153200K .......... .......... .......... .......... .......... 90% 45.7M 1s 2024-03-29T04:08:47.337 INFO:teuthology.orchestra.run.smithi097.stderr:153250K .......... .......... .......... .......... .......... 90% 32.9M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153300K .......... .......... .......... .......... .......... 90% 28.0M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153350K .......... .......... .......... .......... .......... 91% 13.8M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153400K .......... .......... .......... .......... .......... 91% 11.3M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153450K .......... .......... .......... .......... .......... 91% 74.8M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153500K .......... .......... .......... .......... .......... 91% 68.7M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153550K .......... .......... .......... .......... .......... 91% 50.1M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153600K .......... .......... .......... .......... .......... 91% 97.0M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153650K .......... .......... .......... .......... .......... 91% 109M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153700K .......... .......... .......... .......... .......... 91% 81.7M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153750K .......... .......... .......... .......... .......... 91% 11.7M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153800K .......... .......... .......... .......... .......... 91% 82.7M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153850K .......... .......... .......... .......... .......... 91% 46.4M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153900K .......... .......... .......... .......... .......... 91% 84.3M 1s 2024-03-29T04:08:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:153950K .......... .......... .......... .......... .......... 91% 46.4M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154000K .......... .......... .......... .......... .......... 91% 94.4M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154050K .......... .......... .......... .......... .......... 91% 64.2M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154100K .......... .......... .......... .......... .......... 91% 36.9M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154150K .......... .......... .......... .......... .......... 91% 35.1M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154200K .......... .......... .......... .......... .......... 91% 43.6M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154250K .......... .......... .......... .......... .......... 91% 30.4M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154300K .......... .......... .......... .......... .......... 91% 23.8M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154350K .......... .......... .......... .......... .......... 91% 30.6M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154400K .......... .......... .......... .......... .......... 91% 22.0M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154450K .......... .......... .......... .......... .......... 91% 28.8M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154500K .......... .......... .......... .......... .......... 91% 61.5M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154550K .......... .......... .......... .......... .......... 91% 22.6M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154600K .......... .......... .......... .......... .......... 91% 32.1M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154650K .......... .......... .......... .......... .......... 91% 30.4M 1s 2024-03-29T04:08:47.339 INFO:teuthology.orchestra.run.smithi097.stderr:154700K .......... .......... .......... .......... .......... 91% 22.7M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:154750K .......... .......... .......... .......... .......... 91% 27.5M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:154800K .......... .......... .......... .......... .......... 91% 37.1M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:154850K .......... .......... .......... .......... .......... 91% 32.3M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:154900K .......... .......... .......... .......... .......... 91% 26.2M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:154950K .......... .......... .......... .......... .......... 91% 28.3M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:155000K .......... .......... .......... .......... .......... 91% 23.3M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:155050K .......... .......... .......... .......... .......... 92% 38.5M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:155100K .......... .......... .......... .......... .......... 92% 29.6M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:155150K .......... .......... .......... .......... .......... 92% 31.4M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:155200K .......... .......... .......... .......... .......... 92% 23.9M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:155250K .......... .......... .......... .......... .......... 92% 22.6M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:155300K .......... .......... .......... .......... .......... 92% 51.3M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:155350K .......... .......... .......... .......... .......... 92% 24.4M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:155400K .......... .......... .......... .......... .......... 92% 38.2M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:155450K .......... .......... .......... .......... .......... 92% 23.2M 1s 2024-03-29T04:08:47.340 INFO:teuthology.orchestra.run.smithi097.stderr:155500K .......... .......... .......... .......... .......... 92% 38.9M 1s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:155550K .......... .......... .......... .......... .......... 92% 25.3M 1s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:155600K .......... .......... .......... .......... .......... 92% 26.3M 1s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:155650K .......... .......... .......... .......... .......... 92% 15.0M 1s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:155700K .......... .......... .......... .......... .......... 92% 13.5M 1s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:155750K .......... .......... .......... .......... .......... 92% 113M 1s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:155800K .......... .......... .......... .......... .......... 92% 103M 1s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:155850K .......... .......... .......... .......... .......... 92% 62.6M 1s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:155900K .......... .......... .......... .......... .......... 92% 52.2M 1s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:155950K .......... .......... .......... .......... .......... 92% 20.2M 0s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:156000K .......... .......... .......... .......... .......... 92% 52.5M 0s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:156050K .......... .......... .......... .......... .......... 92% 21.7M 0s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:156100K .......... .......... .......... .......... .......... 92% 39.3M 0s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:156150K .......... .......... .......... .......... .......... 92% 25.7M 0s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:156200K .......... .......... .......... .......... .......... 92% 20.8M 0s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:156250K .......... .......... .......... .......... .......... 92% 34.2M 0s 2024-03-29T04:08:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:156300K .......... .......... .......... .......... .......... 92% 26.1M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:156350K .......... .......... .......... .......... .......... 92% 33.8M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:156400K .......... .......... .......... .......... .......... 92% 37.6M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:156450K .......... .......... .......... .......... .......... 92% 22.0M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:156500K .......... .......... .......... .......... .......... 92% 36.0M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:156550K .......... .......... .......... .......... .......... 92% 23.2M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:156600K .......... .......... .......... .......... .......... 92% 33.2M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:156650K .......... .......... .......... .......... .......... 92% 29.5M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:156700K .......... .......... .......... .......... .......... 92% 18.6M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:156750K .......... .......... .......... .......... .......... 93% 36.5M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:156800K .......... .......... .......... .......... .......... 93% 21.4M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:156850K .......... .......... .......... .......... .......... 93% 20.9M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:156900K .......... .......... .......... .......... .......... 93% 41.7M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:156950K .......... .......... .......... .......... .......... 93% 45.7M 0s 2024-03-29T04:08:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:157000K .......... .......... .......... .......... .......... 93% 54.1M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157050K .......... .......... .......... .......... .......... 93% 47.0M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157100K .......... .......... .......... .......... .......... 93% 23.7M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157150K .......... .......... .......... .......... .......... 93% 26.9M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157200K .......... .......... .......... .......... .......... 93% 35.9M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157250K .......... .......... .......... .......... .......... 93% 26.3M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157300K .......... .......... .......... .......... .......... 93% 38.6M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157350K .......... .......... .......... .......... .......... 93% 23.2M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157400K .......... .......... .......... .......... .......... 93% 28.8M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157450K .......... .......... .......... .......... .......... 93% 37.1M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157500K .......... .......... .......... .......... .......... 93% 23.5M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157550K .......... .......... .......... .......... .......... 93% 39.4M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157600K .......... .......... .......... .......... .......... 93% 26.6M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157650K .......... .......... .......... .......... .......... 93% 23.8M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157700K .......... .......... .......... .......... .......... 93% 41.6M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157750K .......... .......... .......... .......... .......... 93% 23.3M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157800K .......... .......... .......... .......... .......... 93% 39.0M 0s 2024-03-29T04:08:47.343 INFO:teuthology.orchestra.run.smithi097.stderr:157850K .......... .......... .......... .......... .......... 93% 24.2M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:157900K .......... .......... .......... .......... .......... 93% 24.1M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:157950K .......... .......... .......... .......... .......... 93% 38.4M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:158000K .......... .......... .......... .......... .......... 93% 26.6M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:158050K .......... .......... .......... .......... .......... 93% 42.0M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:158100K .......... .......... .......... .......... .......... 93% 25.0M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:158150K .......... .......... .......... .......... .......... 93% 23.7M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:158200K .......... .......... .......... .......... .......... 93% 48.4M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:158250K .......... .......... .......... .......... .......... 93% 19.1M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:158300K .......... .......... .......... .......... .......... 93% 58.4M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:158350K .......... .......... .......... .......... .......... 93% 24.3M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:158400K .......... .......... .......... .......... .......... 93% 36.3M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:158450K .......... .......... .......... .......... .......... 94% 25.0M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:158500K .......... .......... .......... .......... .......... 94% 23.0M 0s 2024-03-29T04:08:47.344 INFO:teuthology.orchestra.run.smithi097.stderr:158550K .......... .......... .......... .......... .......... 94% 41.4M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:158600K .......... .......... .......... .......... .......... 94% 22.2M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:158650K .......... .......... .......... .......... .......... 94% 46.8M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:158700K .......... .......... .......... .......... .......... 94% 26.5M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:158750K .......... .......... .......... .......... .......... 94% 32.0M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:158800K .......... .......... .......... .......... .......... 94% 19.7M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:158850K .......... .......... .......... .......... .......... 94% 33.9M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:158900K .......... .......... .......... .......... .......... 94% 48.9M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:158950K .......... .......... .......... .......... .......... 94% 26.9M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:159000K .......... .......... .......... .......... .......... 94% 22.6M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:159050K .......... .......... .......... .......... .......... 94% 27.9M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:159100K .......... .......... .......... .......... .......... 94% 31.5M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:159150K .......... .......... .......... .......... .......... 94% 49.1M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:159200K .......... .......... .......... .......... .......... 94% 24.1M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:159250K .......... .......... .......... .......... .......... 94% 25.4M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:159300K .......... .......... .......... .......... .......... 94% 31.3M 0s 2024-03-29T04:08:47.345 INFO:teuthology.orchestra.run.smithi097.stderr:159350K .......... .......... .......... .......... .......... 94% 25.1M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:159400K .......... .......... .......... .......... .......... 94% 49.3M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:159450K .......... .......... .......... .......... .......... 94% 25.2M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:159500K .......... .......... .......... .......... .......... 94% 23.6M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:159550K .......... .......... .......... .......... .......... 94% 39.9M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:159600K .......... .......... .......... .......... .......... 94% 25.1M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:159650K .......... .......... .......... .......... .......... 94% 24.5M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:159700K .......... .......... .......... .......... .......... 94% 47.6M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:159750K .......... .......... .......... .......... .......... 94% 21.6M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:159800K .......... .......... .......... .......... .......... 94% 28.4M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:159850K .......... .......... .......... .......... .......... 94% 38.9M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:159900K .......... .......... .......... .......... .......... 94% 24.8M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:159950K .......... .......... .......... .......... .......... 94% 38.3M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:160000K .......... .......... .......... .......... .......... 94% 24.0M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:160050K .......... .......... .......... .......... .......... 94% 38.6M 0s 2024-03-29T04:08:47.346 INFO:teuthology.orchestra.run.smithi097.stderr:160100K .......... .......... .......... .......... .......... 95% 26.0M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160150K .......... .......... .......... .......... .......... 95% 23.5M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160200K .......... .......... .......... .......... .......... 95% 35.9M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160250K .......... .......... .......... .......... .......... 95% 23.3M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160300K .......... .......... .......... .......... .......... 95% 38.8M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160350K .......... .......... .......... .......... .......... 95% 24.2M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160400K .......... .......... .......... .......... .......... 95% 28.3M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160450K .......... .......... .......... .......... .......... 95% 46.2M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160500K .......... .......... .......... .......... .......... 95% 21.0M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160550K .......... .......... .......... .......... .......... 95% 45.5M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160600K .......... .......... .......... .......... .......... 95% 20.0M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160650K .......... .......... .......... .......... .......... 95% 19.1M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160700K .......... .......... .......... .......... .......... 95% 33.9M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160750K .......... .......... .......... .......... .......... 95% 18.5M 0s 2024-03-29T04:08:47.347 INFO:teuthology.orchestra.run.smithi097.stderr:160800K .......... .......... .......... .......... .......... 95% 35.2M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:160850K .......... .......... .......... .......... .......... 95% 20.0M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:160900K .......... .......... .......... .......... .......... 95% 28.4M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:160950K .......... .......... .......... .......... .......... 95% 22.9M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:161000K .......... .......... .......... .......... .......... 95% 24.6M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:161050K .......... .......... .......... .......... .......... 95% 19.3M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:161100K .......... .......... .......... .......... .......... 95% 11.3M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:161150K .......... .......... .......... .......... .......... 95% 30.7M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:161200K .......... .......... .......... .......... .......... 95% 23.1M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:161250K .......... .......... .......... .......... .......... 95% 22.2M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:161300K .......... .......... .......... .......... .......... 95% 24.3M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:161350K .......... .......... .......... .......... .......... 95% 14.9M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:161400K .......... .......... .......... .......... .......... 95% 17.8M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:161450K .......... .......... .......... .......... .......... 95% 38.9M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:161500K .......... .......... .......... .......... .......... 95% 17.8M 0s 2024-03-29T04:08:47.348 INFO:teuthology.orchestra.run.smithi097.stderr:161550K .......... .......... .......... .......... .......... 95% 20.0M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:161600K .......... .......... .......... .......... .......... 95% 19.1M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:161650K .......... .......... .......... .......... .......... 95% 13.1M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:161700K .......... .......... .......... .......... .......... 95% 27.9M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:161750K .......... .......... .......... .......... .......... 95% 20.7M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:161800K .......... .......... .......... .......... .......... 96% 15.3M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:161850K .......... .......... .......... .......... .......... 96% 16.5M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:161900K .......... .......... .......... .......... .......... 96% 15.7M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:161950K .......... .......... .......... .......... .......... 96% 35.9M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:162000K .......... .......... .......... .......... .......... 96% 20.7M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:162050K .......... .......... .......... .......... .......... 96% 24.9M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:162100K .......... .......... .......... .......... .......... 96% 16.1M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:162150K .......... .......... .......... .......... .......... 96% 19.1M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:162200K .......... .......... .......... .......... .......... 96% 29.3M 0s 2024-03-29T04:08:47.349 INFO:teuthology.orchestra.run.smithi097.stderr:162250K .......... .......... .......... .......... .......... 96% 26.9M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162300K .......... .......... .......... .......... .......... 96% 19.6M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162350K .......... .......... .......... .......... .......... 96% 34.1M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162400K .......... .......... .......... .......... .......... 96% 21.2M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162450K .......... .......... .......... .......... .......... 96% 54.1M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162500K .......... .......... .......... .......... .......... 96% 24.4M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162550K .......... .......... .......... .......... .......... 96% 28.6M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162600K .......... .......... .......... .......... .......... 96% 35.9M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162650K .......... .......... .......... .......... .......... 96% 33.4M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162700K .......... .......... .......... .......... .......... 96% 23.3M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162750K .......... .......... .......... .......... .......... 96% 23.5M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162800K .......... .......... .......... .......... .......... 96% 27.2M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162850K .......... .......... .......... .......... .......... 96% 27.8M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162900K .......... .......... .......... .......... .......... 96% 31.6M 0s 2024-03-29T04:08:47.350 INFO:teuthology.orchestra.run.smithi097.stderr:162950K .......... .......... .......... .......... .......... 96% 26.7M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163000K .......... .......... .......... .......... .......... 96% 29.1M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163050K .......... .......... .......... .......... .......... 96% 23.7M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163100K .......... .......... .......... .......... .......... 96% 23.7M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163150K .......... .......... .......... .......... .......... 96% 24.5M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163200K .......... .......... .......... .......... .......... 96% 31.8M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163250K .......... .......... .......... .......... .......... 96% 34.0M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163300K .......... .......... .......... .......... .......... 96% 34.6M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163350K .......... .......... .......... .......... .......... 96% 53.3M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163400K .......... .......... .......... .......... .......... 96% 21.0M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163450K .......... .......... .......... .......... .......... 96% 34.5M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163500K .......... .......... .......... .......... .......... 97% 22.8M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163550K .......... .......... .......... .......... .......... 97% 28.8M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163600K .......... .......... .......... .......... .......... 97% 21.8M 0s 2024-03-29T04:08:47.351 INFO:teuthology.orchestra.run.smithi097.stderr:163650K .......... .......... .......... .......... .......... 97% 33.6M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:163700K .......... .......... .......... .......... .......... 97% 36.5M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:163750K .......... .......... .......... .......... .......... 97% 35.9M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:163800K .......... .......... .......... .......... .......... 97% 30.2M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:163850K .......... .......... .......... .......... .......... 97% 27.3M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:163900K .......... .......... .......... .......... .......... 97% 24.9M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:163950K .......... .......... .......... .......... .......... 97% 34.0M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:164000K .......... .......... .......... .......... .......... 97% 27.9M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:164050K .......... .......... .......... .......... .......... 97% 30.9M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:164100K .......... .......... .......... .......... .......... 97% 19.8M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:164150K .......... .......... .......... .......... .......... 97% 28.8M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:164200K .......... .......... .......... .......... .......... 97% 28.5M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:164250K .......... .......... .......... .......... .......... 97% 21.9M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:164300K .......... .......... .......... .......... .......... 97% 27.3M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:164350K .......... .......... .......... .......... .......... 97% 19.7M 0s 2024-03-29T04:08:47.352 INFO:teuthology.orchestra.run.smithi097.stderr:164400K .......... .......... .......... .......... .......... 97% 23.1M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:164450K .......... .......... .......... .......... .......... 97% 34.9M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:164500K .......... .......... .......... .......... .......... 97% 27.0M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:164550K .......... .......... .......... .......... .......... 97% 24.7M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:164600K .......... .......... .......... .......... .......... 97% 28.8M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:164650K .......... .......... .......... .......... .......... 97% 20.5M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:164700K .......... .......... .......... .......... .......... 97% 59.8M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:164750K .......... .......... .......... .......... .......... 97% 21.7M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:164800K .......... .......... .......... .......... .......... 97% 22.3M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:164850K .......... .......... .......... .......... .......... 97% 34.6M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:164900K .......... .......... .......... .......... .......... 97% 22.1M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:164950K .......... .......... .......... .......... .......... 97% 30.0M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:165000K .......... .......... .......... .......... .......... 97% 29.3M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:165050K .......... .......... .......... .......... .......... 97% 25.1M 0s 2024-03-29T04:08:47.353 INFO:teuthology.orchestra.run.smithi097.stderr:165100K .......... .......... .......... .......... .......... 97% 31.4M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165150K .......... .......... .......... .......... .......... 98% 21.5M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165200K .......... .......... .......... .......... .......... 98% 40.4M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165250K .......... .......... .......... .......... .......... 98% 32.7M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165300K .......... .......... .......... .......... .......... 98% 32.1M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165350K .......... .......... .......... .......... .......... 98% 22.4M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165400K .......... .......... .......... .......... .......... 98% 39.1M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165450K .......... .......... .......... .......... .......... 98% 48.7M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165500K .......... .......... .......... .......... .......... 98% 39.9M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165550K .......... .......... .......... .......... .......... 98% 23.3M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165600K .......... .......... .......... .......... .......... 98% 39.7M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165650K .......... .......... .......... .......... .......... 98% 34.6M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165700K .......... .......... .......... .......... .......... 98% 46.0M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165750K .......... .......... .......... .......... .......... 98% 23.4M 0s 2024-03-29T04:08:47.354 INFO:teuthology.orchestra.run.smithi097.stderr:165800K .......... .......... .......... .......... .......... 98% 37.4M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:165850K .......... .......... .......... .......... .......... 98% 35.1M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:165900K .......... .......... .......... .......... .......... 98% 23.1M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:165950K .......... .......... .......... .......... .......... 98% 30.4M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:166000K .......... .......... .......... .......... .......... 98% 32.8M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:166050K .......... .......... .......... .......... .......... 98% 28.6M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:166100K .......... .......... .......... .......... .......... 98% 50.0M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:166150K .......... .......... .......... .......... .......... 98% 31.4M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:166200K .......... .......... .......... .......... .......... 98% 39.5M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:166250K .......... .......... .......... .......... .......... 98% 33.3M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:166300K .......... .......... .......... .......... .......... 98% 27.7M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:166350K .......... .......... .......... .......... .......... 98% 33.7M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:166400K .......... .......... .......... .......... .......... 98% 29.9M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:166450K .......... .......... .......... .......... .......... 98% 48.0M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:166500K .......... .......... .......... .......... .......... 98% 29.8M 0s 2024-03-29T04:08:47.355 INFO:teuthology.orchestra.run.smithi097.stderr:166550K .......... .......... .......... .......... .......... 98% 24.5M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:166600K .......... .......... .......... .......... .......... 98% 38.1M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:166650K .......... .......... .......... .......... .......... 98% 22.5M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:166700K .......... .......... .......... .......... .......... 98% 84.2M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:166750K .......... .......... .......... .......... .......... 98% 26.6M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:166800K .......... .......... .......... .......... .......... 98% 28.5M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:166850K .......... .......... .......... .......... .......... 99% 33.9M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:166900K .......... .......... .......... .......... .......... 99% 39.4M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:166950K .......... .......... .......... .......... .......... 99% 23.0M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:167000K .......... .......... .......... .......... .......... 99% 36.7M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:167050K .......... .......... .......... .......... .......... 99% 21.3M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:167100K .......... .......... .......... .......... .......... 99% 31.1M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:167150K .......... .......... .......... .......... .......... 99% 34.6M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:167200K .......... .......... .......... .......... .......... 99% 20.2M 0s 2024-03-29T04:08:47.356 INFO:teuthology.orchestra.run.smithi097.stderr:167250K .......... .......... .......... .......... .......... 99% 30.9M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167300K .......... .......... .......... .......... .......... 99% 27.1M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167350K .......... .......... .......... .......... .......... 99% 19.5M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167400K .......... .......... .......... .......... .......... 99% 41.3M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167450K .......... .......... .......... .......... .......... 99% 27.6M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167500K .......... .......... .......... .......... .......... 99% 31.1M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167550K .......... .......... .......... .......... .......... 99% 29.9M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167600K .......... .......... .......... .......... .......... 99% 21.2M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167650K .......... .......... .......... .......... .......... 99% 32.7M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167700K .......... .......... .......... .......... .......... 99% 31.1M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167750K .......... .......... .......... .......... .......... 99% 30.9M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167800K .......... .......... .......... .......... .......... 99% 30.2M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167850K .......... .......... .......... .......... .......... 99% 30.6M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167900K .......... .......... .......... .......... .......... 99% 37.6M 0s 2024-03-29T04:08:47.357 INFO:teuthology.orchestra.run.smithi097.stderr:167950K .......... .......... .......... .......... .......... 99% 17.3M 0s 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:168000K .......... .......... .......... .......... .......... 99% 25.9M 0s 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:168050K .......... .......... .......... .......... .......... 99% 16.1M 0s 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:168100K .......... .......... .......... .......... .......... 99% 30.2M 0s 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:168150K .......... .......... .......... .......... .......... 99% 42.2M 0s 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:168200K .......... .......... .......... .......... .......... 99% 24.1M 0s 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:168250K .......... .......... .......... .......... .......... 99% 49.7M 0s 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:168300K .......... .......... .......... .......... .......... 99% 19.1M 0s 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:168350K .......... .......... .......... .......... .......... 99% 21.1M 0s 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:168400K .......... .......... .......... .......... .......... 99% 30.4M 0s 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:168450K .......... .......... .......... .......... .......... 99% 32.5M 0s 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:168500K .......... .......... .......... .......... .......... 99% 20.3M 0s 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:168550K .......... .......... . 100% 115M=6.6s 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29 04:08:46 (24.8 MB/s) - ‘linux-5.4.tar.gz’ saved [172616875/172616875] 2024-03-29T04:08:47.358 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-29T04:08:47.359 INFO:tasks.cephfs.test_uninlining:Running Data Uninlining test with: without clearing cache, without unmount client, without mount.open_background, without multimds, without snapshots 2024-03-29T04:08:47.359 DEBUG:tasks.cephfs.test_uninlining:setting max_mds:1 2024-03-29T04:08:47.359 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-03-29T04:08:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.493+0000 7f1746dc8640 1 Processor -- start 2024-03-29T04:08:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.493+0000 7f1746dc8640 1 -- start start 2024-03-29T04:08:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.494+0000 7f1746dc8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f174015b3d0 0x7f17400ca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.494+0000 7f1746dc8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17400c6470 con 0x7f17400b28c0 2024-03-29T04:08:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.494+0000 7f1746dc8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17400c6750 con 0x7f17400b49f0 2024-03-29T04:08:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.494+0000 7f1746dc8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f17400c6240 con 0x7f174015b3d0 2024-03-29T04:08:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.494+0000 7f1744b64640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f174015b3d0 0x7f17400ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.494+0000 7f1745365640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f17400b28c0 0x7f174015be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35966/0 (socket says 172.21.15.97:35966) 2024-03-29T04:08:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.494+0000 7f1745365640 1 -- 172.21.15.97:0/678971700 learned_addr learned my addr 172.21.15.97:0/678971700 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.495+0000 7f1744b64640 1 -- 172.21.15.97:0/678971700 >> v1:172.21.15.154:6789/0 conn(0x7f17400b49f0 legacy=0x7f17400b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.495+0000 7f1744b64640 1 -- 172.21.15.97:0/678971700 >> v1:172.21.15.97:6789/0 conn(0x7f17400b28c0 legacy=0x7f174015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.495+0000 7f1744b64640 1 -- 172.21.15.97:0/678971700 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17400c6a30 con 0x7f174015b3d0 2024-03-29T04:08:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.495+0000 7f173f7fe640 1 -- 172.21.15.97:0/678971700 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 454279499 0 0) 0x7f17400c6470 con 0x7f17400b28c0 2024-03-29T04:08:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.495+0000 7f1744b64640 1 --2- 172.21.15.97:0/678971700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174015b3d0 0x7f17400ca950 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f17340097a0 tx=0x7f173402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6641665706838b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.496+0000 7f173f7fe640 1 -- 172.21.15.97:0/678971700 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734035020 con 0x7f174015b3d0 2024-03-29T04:08:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.496+0000 7f173f7fe640 1 -- 172.21.15.97:0/678971700 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1734031830 con 0x7f174015b3d0 2024-03-29T04:08:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.496+0000 7f173f7fe640 1 -- 172.21.15.97:0/678971700 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734031c00 con 0x7f174015b3d0 2024-03-29T04:08:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.496+0000 7f1746dc8640 1 -- 172.21.15.97:0/678971700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174015b3d0 msgr2=0x7f17400ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.496+0000 7f1746dc8640 1 --2- 172.21.15.97:0/678971700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174015b3d0 0x7f17400ca950 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f17340097a0 tx=0x7f173402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.496+0000 7f1746dc8640 1 -- 172.21.15.97:0/678971700 shutdown_connections 2024-03-29T04:08:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.496+0000 7f1746dc8640 1 --2- 172.21.15.97:0/678971700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f174015b3d0 0x7f17400ca950 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.496+0000 7f1746dc8640 1 -- 172.21.15.97:0/678971700 >> 172.21.15.97:0/678971700 conn(0x7f17400d0450 msgr2=0x7f17400d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.496+0000 7f1746dc8640 1 -- 172.21.15.97:0/678971700 shutdown_connections 2024-03-29T04:08:47.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.496+0000 7f1746dc8640 1 -- 172.21.15.97:0/678971700 wait complete. 2024-03-29T04:08:47.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.497+0000 7f1746dc8640 1 Processor -- start 2024-03-29T04:08:47.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.497+0000 7f1746dc8640 1 -- start start 2024-03-29T04:08:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.498+0000 7f1746dc8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17400b28c0 0x7f17400717d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.498+0000 7f1746dc8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17400b49f0 0x7f1740153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.498+0000 7f1746dc8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f174015b3d0 0x7f174015dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.498+0000 7f1746dc8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f174015b7b0 con 0x7f17400b49f0 2024-03-29T04:08:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.498+0000 7f1746dc8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f17400c7080 con 0x7f174015b3d0 2024-03-29T04:08:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.498+0000 7f1746dc8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f174015b930 con 0x7f17400b28c0 2024-03-29T04:08:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.498+0000 7f173ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17400b49f0 0x7f1740153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.498+0000 7f173ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17400b49f0 0x7f1740153930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52228/0 (socket says 172.21.15.97:52228) 2024-03-29T04:08:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.498+0000 7f173ffff640 1 -- 172.21.15.97:0/2633261522 learned_addr learned my addr 172.21.15.97:0/2633261522 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.498+0000 7f1744b64640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17400b28c0 0x7f17400717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.498+0000 7f1745365640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f174015b3d0 0x7f174015dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.499+0000 7f173ffff640 1 -- 172.21.15.97:0/2633261522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17400b28c0 msgr2=0x7f17400717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.499+0000 7f173ffff640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17400b28c0 0x7f17400717d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:47.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.499+0000 7f173ffff640 1 -- 172.21.15.97:0/2633261522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f174015b3d0 msgr2=0x7f174015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.499+0000 7f173ffff640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f174015b3d0 0x7f174015dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.499+0000 7f173ffff640 1 -- 172.21.15.97:0/2633261522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17401f1da0 con 0x7f17400b49f0 2024-03-29T04:08:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.499+0000 7f173ffff640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17400b49f0 0x7f1740153930 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f172c00c410 tx=0x7f172c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.499+0000 7f173d7fa640 1 -- 172.21.15.97:0/2633261522 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c017070 con 0x7f17400b49f0 2024-03-29T04:08:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.499+0000 7f173d7fa640 1 -- 172.21.15.97:0/2633261522 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172c0035f0 con 0x7f17400b49f0 2024-03-29T04:08:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.499+0000 7f1746dc8640 1 -- 172.21.15.97:0/2633261522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17401f2050 con 0x7f17400b49f0 2024-03-29T04:08:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.499+0000 7f173d7fa640 1 -- 172.21.15.97:0/2633261522 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c003a20 con 0x7f17400b49f0 2024-03-29T04:08:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.499+0000 7f1746dc8640 1 -- 172.21.15.97:0/2633261522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17401f2610 con 0x7f17400b49f0 2024-03-29T04:08:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.500+0000 7f1746dc8640 1 -- 172.21.15.97:0/2633261522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174009e520 con 0x7f17400b49f0 2024-03-29T04:08:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.500+0000 7f173d7fa640 1 -- 172.21.15.97:0/2633261522 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f172c01a070 con 0x7f17400b49f0 2024-03-29T04:08:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.501+0000 7f173d7fa640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f172406c530 0x7f172406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.501+0000 7f1744b64640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f172406c530 0x7f172406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:47.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.501+0000 7f173d7fa640 1 -- 172.21.15.97:0/2633261522 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f172c08ab30 con 0x7f17400b49f0 2024-03-29T04:08:47.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.502+0000 7f1744b64640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f172406c530 0x7f172406e9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1734036000 tx=0x7f1734008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:47.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.506+0000 7f173d7fa640 1 -- 172.21.15.97:0/2633261522 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f172c0591e0 con 0x7f17400b49f0 2024-03-29T04:08:47.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:47.763+0000 7f1746dc8640 1 -- 172.21.15.97:0/2633261522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) -- 0x7f174004f9d0 con 0x7f17400b49f0 2024-03-29T04:08:48.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:48.263+0000 7f173d7fa640 1 -- 172.21.15.97:0/2633261522 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f172c0010c0 con 0x7f17400b49f0 2024-03-29T04:08:48.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:48.265+0000 7f1746dc8640 1 -- 172.21.15.97:0/2633261522 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f172406c530 msgr2=0x7f172406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:48.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:48.265+0000 7f1746dc8640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f172406c530 0x7f172406e9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1734036000 tx=0x7f1734008030 comp rx=0 tx=0).stop 2024-03-29T04:08:48.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:48.265+0000 7f1746dc8640 1 -- 172.21.15.97:0/2633261522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17400b49f0 msgr2=0x7f1740153930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:48.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:48.265+0000 7f1746dc8640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17400b49f0 0x7f1740153930 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f172c00c410 tx=0x7f172c00c8d0 comp rx=0 tx=0).stop 2024-03-29T04:08:48.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:48.265+0000 7f1746dc8640 1 -- 172.21.15.97:0/2633261522 shutdown_connections 2024-03-29T04:08:48.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:48.265+0000 7f1746dc8640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f172406c530 0x7f172406e9f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:48.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:48.265+0000 7f1746dc8640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f174015b3d0 0x7f174015dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:48.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:48.265+0000 7f1746dc8640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17400b49f0 0x7f1740153930 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:48.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:48.265+0000 7f1746dc8640 1 --2- 172.21.15.97:0/2633261522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17400b28c0 0x7f17400717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:48.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:48.265+0000 7f1746dc8640 1 -- 172.21.15.97:0/2633261522 >> 172.21.15.97:0/2633261522 conn(0x7f17400d0450 msgr2=0x7f174006de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:48.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:48.265+0000 7f1746dc8640 1 -- 172.21.15.97:0/2633261522 shutdown_connections 2024-03-29T04:08:48.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:48.265+0000 7f1746dc8640 1 -- 172.21.15.97:0/2633261522 wait complete. 2024-03-29T04:08:48.257 DEBUG:tasks.cephfs.test_uninlining:getting client_max_inline_size 2024-03-29T04:08:48.257 DEBUG:tasks.cephfs.test_uninlining:got client_max_inline_size:4096 2024-03-29T04:08:48.257 DEBUG:tasks.cephfs.test_uninlining:extracting tarball 2024-03-29T04:08:48.257 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && tar -x -z -f linux-5.4.tar.gz linux-5.4/fs/ceph linux-5.4/fs/orangefs linux-5.4/fs/ext2 2024-03-29T04:08:52.481 DEBUG:tasks.cephfs.test_uninlining:fetching dir list 2024-03-29T04:08:52.481 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -mindepth 2 -maxdepth 2 -type d 2024-03-29T04:08:52.512 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ceph 2024-03-29T04:08:52.513 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ext2 2024-03-29T04:08:52.513 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/orangefs 2024-03-29T04:08:52.513 DEBUG:tasks.cephfs.test_uninlining:distributing dir pins 2024-03-29T04:08:52.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:52.513 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ceph) 2024-03-29T04:08:52.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:52.578 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ext2) 2024-03-29T04:08:52.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:52.634 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/orangefs) 2024-03-29T04:08:52.690 DEBUG:tasks.cephfs.test_uninlining:listing files with size <= 4096 2024-03-29T04:08:52.690 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -type f -size -4097c 2024-03-29T04:08:52.761 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ceph/Kconfig 2024-03-29T04:08:52.761 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ceph/Makefile 2024-03-29T04:08:52.761 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:08:52.761 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ceph/io.h 2024-03-29T04:08:52.761 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:08:52.761 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ext2/Kconfig 2024-03-29T04:08:52.761 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:08:52.761 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:08:52.761 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ext2/Makefile 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ext2/symlink.c 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ext2/acl.h 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/ext2/xattr.h 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/orangefs/Makefile 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:08:52.762 INFO:teuthology.orchestra.run.smithi097.stdout:linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:08:52.763 INFO:tasks.cephfs.test_uninlining:Found 23 inlined files 2024-03-29T04:08:52.763 DEBUG:tasks.cephfs.test_uninlining:starting scrub 2024-03-29T04:08:52.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:52.763 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive 2024-03-29T04:08:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.928+0000 7f2f4ab82640 1 Processor -- start 2024-03-29T04:08:52.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.928+0000 7f2f4ab82640 1 -- start start 2024-03-29T04:08:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.928+0000 7f2f4ab82640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f44156ba0 0x7f2f4415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.928+0000 7f2f4ab82640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f4406d860 0x7f2f4406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.928+0000 7f2f4ab82640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f440d0490 0x7f2f440d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.928+0000 7f2f4ab82640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2f4415db70 con 0x7f2f44156ba0 2024-03-29T04:08:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.928+0000 7f2f4ab82640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f2f4406e300 con 0x7f2f440d0490 2024-03-29T04:08:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.928+0000 7f2f4ab82640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2f4406e5e0 con 0x7f2f4406d860 2024-03-29T04:08:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.929+0000 7f2f4891e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f44156ba0 0x7f2f4415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.929+0000 7f2f4891e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f44156ba0 0x7f2f4415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59830/0 (socket says 172.21.15.97:59830) 2024-03-29T04:08:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.929+0000 7f2f4891e640 1 -- 172.21.15.97:0/3372655692 learned_addr learned my addr 172.21.15.97:0/3372655692 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.929+0000 7f2f43fff640 1 --2- 172.21.15.97:0/3372655692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f4406d860 0x7f2f4406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.929+0000 7f2f4911f640 1 --2- 172.21.15.97:0/3372655692 >> v2:172.21.15.154:3300/0 conn(0x7f2f440d0490 0x7f2f440d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.929+0000 7f2f4891e640 1 -- 172.21.15.97:0/3372655692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f4406d860 msgr2=0x7f2f4406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.929+0000 7f2f4891e640 1 --2- 172.21.15.97:0/3372655692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f4406d860 0x7f2f4406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.929+0000 7f2f4891e640 1 -- 172.21.15.97:0/3372655692 >> v2:172.21.15.154:3300/0 conn(0x7f2f440d0490 msgr2=0x7f2f440d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.929+0000 7f2f4891e640 1 --2- 172.21.15.97:0/3372655692 >> v2:172.21.15.154:3300/0 conn(0x7f2f440d0490 0x7f2f440d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.929+0000 7f2f4891e640 1 -- 172.21.15.97:0/3372655692 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4415ce50 con 0x7f2f44156ba0 2024-03-29T04:08:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.929+0000 7f2f4911f640 1 --2- 172.21.15.97:0/3372655692 >> v2:172.21.15.154:3300/0 conn(0x7f2f440d0490 0x7f2f440d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.929+0000 7f2f43fff640 1 --2- 172.21.15.97:0/3372655692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f4406d860 0x7f2f4406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.930+0000 7f2f4891e640 1 --2- 172.21.15.97:0/3372655692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f44156ba0 0x7f2f4415d630 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f4409e4d0 tx=0x7f2f3402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee321ef180564cc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.930+0000 7f2f437fe640 1 -- 172.21.15.97:0/3372655692 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3402e070 con 0x7f2f44156ba0 2024-03-29T04:08:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.930+0000 7f2f437fe640 1 -- 172.21.15.97:0/3372655692 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f34037070 con 0x7f2f44156ba0 2024-03-29T04:08:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.930+0000 7f2f437fe640 1 -- 172.21.15.97:0/3372655692 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34031a00 con 0x7f2f44156ba0 2024-03-29T04:08:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.930+0000 7f2f4ab82640 1 -- 172.21.15.97:0/3372655692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f44156ba0 msgr2=0x7f2f4415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.930+0000 7f2f4ab82640 1 --2- 172.21.15.97:0/3372655692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f44156ba0 0x7f2f4415d630 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f4409e4d0 tx=0x7f2f3402b880 comp rx=0 tx=0).stop 2024-03-29T04:08:52.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.930+0000 7f2f4ab82640 1 -- 172.21.15.97:0/3372655692 shutdown_connections 2024-03-29T04:08:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.930+0000 7f2f4ab82640 1 --2- 172.21.15.97:0/3372655692 >> v2:172.21.15.154:3300/0 conn(0x7f2f440d0490 0x7f2f440d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.930+0000 7f2f4ab82640 1 --2- 172.21.15.97:0/3372655692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f4406d860 0x7f2f4406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.930+0000 7f2f4ab82640 1 --2- 172.21.15.97:0/3372655692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f44156ba0 0x7f2f4415d630 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.930+0000 7f2f4ab82640 1 -- 172.21.15.97:0/3372655692 >> 172.21.15.97:0/3372655692 conn(0x7f2f4406b6f0 msgr2=0x7f2f4406bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.930+0000 7f2f4ab82640 1 -- 172.21.15.97:0/3372655692 shutdown_connections 2024-03-29T04:08:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.930+0000 7f2f4ab82640 1 -- 172.21.15.97:0/3372655692 wait complete. 2024-03-29T04:08:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.931+0000 7f2f4ab82640 1 Processor -- start 2024-03-29T04:08:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.932+0000 7f2f4ab82640 1 -- start start 2024-03-29T04:08:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.932+0000 7f2f4ab82640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4406d860 0x7f2f4414e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.932+0000 7f2f4ab82640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f440d0490 0x7f2f4409dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.932+0000 7f2f4ab82640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44156ba0 0x7f2f441536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.932+0000 7f2f4ab82640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2f440ad3d0 con 0x7f2f440d0490 2024-03-29T04:08:52.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.932+0000 7f2f4ab82640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4415cb40 con 0x7f2f4406d860 2024-03-29T04:08:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.932+0000 7f2f4ab82640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2f44156f80 con 0x7f2f44156ba0 2024-03-29T04:08:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.932+0000 7f2f43fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f440d0490 0x7f2f4409dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f43fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f440d0490 0x7f2f4409dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59836/0 (socket says 172.21.15.97:59836) 2024-03-29T04:08:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f43fff640 1 -- 172.21.15.97:0/3735137388 learned_addr learned my addr 172.21.15.97:0/3735137388 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f4911f640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44156ba0 0x7f2f441536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f4891e640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4406d860 0x7f2f4414e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f43fff640 1 -- 172.21.15.97:0/3735137388 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44156ba0 msgr2=0x7f2f441536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f43fff640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44156ba0 0x7f2f441536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f43fff640 1 -- 172.21.15.97:0/3735137388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4406d860 msgr2=0x7f2f4414e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f43fff640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4406d860 0x7f2f4414e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f43fff640 1 -- 172.21.15.97:0/3735137388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4409cce0 con 0x7f2f440d0490 2024-03-29T04:08:52.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f4891e640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4406d860 0x7f2f4414e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f43fff640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f440d0490 0x7f2f4409dd10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00c820 tx=0x7f2f2c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f417fa640 1 -- 172.21.15.97:0/3735137388 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c004230 con 0x7f2f440d0490 2024-03-29T04:08:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f417fa640 1 -- 172.21.15.97:0/3735137388 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f2c0043f0 con 0x7f2f440d0490 2024-03-29T04:08:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f4ab82640 1 -- 172.21.15.97:0/3735137388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4409e250 con 0x7f2f440d0490 2024-03-29T04:08:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f417fa640 1 -- 172.21.15.97:0/3735137388 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c007a70 con 0x7f2f440d0490 2024-03-29T04:08:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.933+0000 7f2f4ab82640 1 -- 172.21.15.97:0/3735137388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4409d230 con 0x7f2f440d0490 2024-03-29T04:08:52.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.935+0000 7f2f417fa640 1 -- 172.21.15.97:0/3735137388 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2f2c007c10 con 0x7f2f440d0490 2024-03-29T04:08:52.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.936+0000 7f2f417fa640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2f2406c530 0x7f2f2406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.936+0000 7f2f417fa640 1 -- 172.21.15.97:0/3735137388 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2f2c08b7d0 con 0x7f2f440d0490 2024-03-29T04:08:52.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.936+0000 7f2f4891e640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2f2406c530 0x7f2f2406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.936+0000 7f2f4891e640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2f2406c530 0x7f2f2406e9f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2f34036000 tx=0x7f2f3403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.936+0000 7f2f4d76a740 1 Processor -- start 2024-03-29T04:08:52.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.936+0000 7f2f4d76a740 1 -- start start 2024-03-29T04:08:52.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.937+0000 7f2f4d76a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e21ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.937+0000 7f2f4d76a740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 0x55ebb8e518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4d76a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e22000 0x55ebb8e51df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4d76a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ebb8d8b410 con 0x55ebb8e4eb20 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4d76a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ebb8d29a80 con 0x55ebb8e22000 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4d76a740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55ebb8d873c0 con 0x55ebb8e4ef00 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4891e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e21ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4891e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e21ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59844/0 (socket says 172.21.15.97:59844) 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4891e640 1 -- 172.21.15.97:0/2661027272 learned_addr learned my addr 172.21.15.97:0/2661027272 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4911f640 1 --2- 172.21.15.97:0/2661027272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e22000 0x55ebb8e51df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f43fff640 1 --2- 172.21.15.97:0/2661027272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 0x55ebb8e518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4891e640 1 -- 172.21.15.97:0/2661027272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 msgr2=0x55ebb8e518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4891e640 1 --2- 172.21.15.97:0/2661027272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 0x55ebb8e518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4891e640 1 -- 172.21.15.97:0/2661027272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e22000 msgr2=0x55ebb8e51df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4891e640 1 --2- 172.21.15.97:0/2661027272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e22000 0x55ebb8e51df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4891e640 1 -- 172.21.15.97:0/2661027272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebb8b6b7b0 con 0x55ebb8e4eb20 2024-03-29T04:08:52.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4911f640 1 --2- 172.21.15.97:0/2661027272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e22000 0x55ebb8e51df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f4891e640 1 --2- 172.21.15.97:0/2661027272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e21ac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009930 tx=0x7f2f34030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77783f36111caa66 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f3a7fc640 1 -- 172.21.15.97:0/2661027272 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34031d10 con 0x55ebb8e4eb20 2024-03-29T04:08:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f3a7fc640 1 -- 172.21.15.97:0/2661027272 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f34037040 con 0x55ebb8e4eb20 2024-03-29T04:08:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.938+0000 7f2f3a7fc640 1 -- 172.21.15.97:0/2661027272 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34031eb0 con 0x55ebb8e4eb20 2024-03-29T04:08:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.939+0000 7f2f4d76a740 1 -- 172.21.15.97:0/2661027272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4eb20 msgr2=0x55ebb8e21ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.939+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/2661027272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e21ac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009930 tx=0x7f2f34030000 comp rx=0 tx=0).stop 2024-03-29T04:08:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.939+0000 7f2f4d76a740 1 -- 172.21.15.97:0/2661027272 shutdown_connections 2024-03-29T04:08:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.939+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/2661027272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e22000 0x55ebb8e51df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.939+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/2661027272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 0x55ebb8e518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.939+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/2661027272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e21ac0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.939+0000 7f2f4d76a740 1 -- 172.21.15.97:0/2661027272 >> 172.21.15.97:0/2661027272 conn(0x55ebb8e23d30 msgr2=0x55ebb8e6c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.939+0000 7f2f4d76a740 1 -- 172.21.15.97:0/2661027272 shutdown_connections 2024-03-29T04:08:52.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.939+0000 7f2f4d76a740 1 -- 172.21.15.97:0/2661027272 wait complete. 2024-03-29T04:08:52.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.939+0000 7f2f4d76a740 1 -- start start 2024-03-29T04:08:52.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.940+0000 7f2f4d76a740 1 Processor -- start 2024-03-29T04:08:52.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.940+0000 7f2f4d76a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e56d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.940+0000 7f2f4d76a740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 0x55ebb8e58230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.940+0000 7f2f4d76a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 0x55ebb8e58770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.940+0000 7f2f4d76a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ebb8e263d0 con 0x55ebb8e572c0 2024-03-29T04:08:52.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.940+0000 7f2f4d76a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ebb8d27740 con 0x55ebb8e4eb20 2024-03-29T04:08:52.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.940+0000 7f2f4d76a740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55ebb8e28c10 con 0x55ebb8e4ef00 2024-03-29T04:08:52.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.940+0000 7f2f4d76a740 10 client.0 _ll_register_callbacks cb 0x55ebb8d34b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:08:52.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.940+0000 7f2f4911f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 0x55ebb8e58770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.940+0000 7f2f4911f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 0x55ebb8e58770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59852/0 (socket says 172.21.15.97:59852) 2024-03-29T04:08:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.940+0000 7f2f4911f640 1 -- 172.21.15.97:0/2066983850 learned_addr learned my addr 172.21.15.97:0/2066983850 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f4891e640 1 --2- 172.21.15.97:0/2066983850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e56d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f43fff640 1 --2- 172.21.15.97:0/2066983850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 0x55ebb8e58230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f4911f640 1 -- 172.21.15.97:0/2066983850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 msgr2=0x55ebb8e58230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f4911f640 1 --2- 172.21.15.97:0/2066983850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 0x55ebb8e58230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f4911f640 1 -- 172.21.15.97:0/2066983850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e4eb20 msgr2=0x55ebb8e56d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f4911f640 1 --2- 172.21.15.97:0/2066983850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e56d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f4911f640 1 -- 172.21.15.97:0/2066983850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebb8bcbf40 con 0x55ebb8e572c0 2024-03-29T04:08:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f4911f640 1 --2- 172.21.15.97:0/2066983850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 0x55ebb8e58770 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00ebb0 tx=0x7f2f3c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c54a58e2156ad2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f237fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-29T04:08:52.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f237fe640 1 -- 172.21.15.97:0/2066983850 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c01c060 con 0x55ebb8e572c0 2024-03-29T04:08:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f237fe640 1 -- 172.21.15.97:0/2066983850 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2f3c00a620 con 0x55ebb8e572c0 2024-03-29T04:08:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f4d76a740 1 -- 172.21.15.97:0/2066983850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f34037040 con 0x55ebb8e572c0 2024-03-29T04:08:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f237fe640 1 -- 172.21.15.97:0/2066983850 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f2f3c022050 con 0x55ebb8e572c0 2024-03-29T04:08:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f4d76a740 10 client.5310 fetch_fsmap learned FSMap version 16 2024-03-29T04:08:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f4d76a740 1 -- 172.21.15.97:0/2066983850 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55ebb8a5f0a0 con 0x55ebb8e572c0 2024-03-29T04:08:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.941+0000 7f2f237fe640 1 -- 172.21.15.97:0/2066983850 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f34037040 con 0x55ebb8e572c0 2024-03-29T04:08:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.942+0000 7f2f237fe640 1 -- 172.21.15.97:0/2066983850 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f2f3c025020 con 0x55ebb8e572c0 2024-03-29T04:08:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.942+0000 7f2f4d76a740 10 client.5310 fetch_fsmap finished waiting for FSMap version 16 2024-03-29T04:08:52.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.942+0000 7f2f4d76a740 10 client.5310 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.942+0000 7f2f4d76a740 20 client.5310 populate_metadata read hostname 'smithi097' 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.942+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/2066983850 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ebb8e7ccf0 0x55ebb8e538b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.942+0000 7f2f4d76a740 4 client.5310 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.942+0000 7f2f4d76a740 1 -- 172.21.15.97:0/2066983850 --> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebb8d98950 con 0x55ebb8e7ccf0 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.942+0000 7f2f4891e640 1 --2- 172.21.15.97:0/2066983850 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ebb8e7ccf0 0x55ebb8e538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.942+0000 7f2f237fe640 10 client.5310 ms_handle_connect on v2:172.21.15.97:6834/3142898446 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.942+0000 7f2f4891e640 1 --2- 172.21.15.97:0/2066983850 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ebb8e7ccf0 0x55ebb8e538b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d2cc77b5907425f server_cookie=8ca58b98de4c010a in_seq=0 out_seq=0 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f237fe640 1 -- 172.21.15.97:0/2066983850 <== mds.0 v2:172.21.15.97:6834/3142898446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55ebb8d98950 con 0x55ebb8e7ccf0 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f237fe640 10 client.5310 handle_command_reply: tid=0 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f4d76a740 1 client.5310 shutdown 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f4d76a740 1 -- 172.21.15.97:0/2066983850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 msgr2=0x55ebb8e58770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/2066983850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 0x55ebb8e58770 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00ebb0 tx=0x7f2f3c00c7f0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f4d76a740 1 -- 172.21.15.97:0/2066983850 shutdown_connections 2024-03-29T04:08:52.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/2066983850 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ebb8e7ccf0 0x55ebb8e538b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/2066983850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 0x55ebb8e58770 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f237fe640 0 client.5310 ms_handle_reset on v2:172.21.15.97:6834/3142898446 2024-03-29T04:08:52.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/2066983850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 0x55ebb8e58230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/2066983850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e56d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f4d76a740 1 -- 172.21.15.97:0/2066983850 >> 172.21.15.97:0/2066983850 conn(0x55ebb8e22000 msgr2=0x55ebb8e25bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f4d76a740 1 -- 172.21.15.97:0/2066983850 shutdown_connections 2024-03-29T04:08:52.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f4d76a740 1 -- 172.21.15.97:0/2066983850 wait complete. 2024-03-29T04:08:52.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.943+0000 7f2f4d76a740 20 client.5310 trim_cache size 0 max 16384 2024-03-29T04:08:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.961+0000 7f2f4d76a740 1 Processor -- start 2024-03-29T04:08:52.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.961+0000 7f2f4d76a740 1 -- start start 2024-03-29T04:08:52.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.961+0000 7f2f4d76a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e27de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.961+0000 7f2f4d76a740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 0x55ebb8e6da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.961+0000 7f2f4d76a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 0x55ebb8e1d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.961+0000 7f2f4d76a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441fe830 con 0x55ebb8e572c0 2024-03-29T04:08:52.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.961+0000 7f2f4d76a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441fda40 con 0x55ebb8e4eb20 2024-03-29T04:08:52.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.961+0000 7f2f4d76a740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2f441f58e0 con 0x55ebb8e4ef00 2024-03-29T04:08:52.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f4911f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 0x55ebb8e1d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f4911f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 0x55ebb8e1d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59858/0 (socket says 172.21.15.97:59858) 2024-03-29T04:08:52.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f4911f640 1 -- 172.21.15.97:0/1325213014 learned_addr learned my addr 172.21.15.97:0/1325213014 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:52.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f43fff640 1 --2- 172.21.15.97:0/1325213014 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 0x55ebb8e6da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f4911f640 1 -- 172.21.15.97:0/1325213014 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 msgr2=0x55ebb8e6da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f4911f640 1 --2- 172.21.15.97:0/1325213014 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 0x55ebb8e6da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f4891e640 1 --2- 172.21.15.97:0/1325213014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e27de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f4911f640 1 -- 172.21.15.97:0/1325213014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e4eb20 msgr2=0x55ebb8e27de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f4911f640 1 --2- 172.21.15.97:0/1325213014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e27de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f4911f640 1 -- 172.21.15.97:0/1325213014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebb8e51db0 con 0x55ebb8e572c0 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f4891e640 1 --2- 172.21.15.97:0/1325213014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e27de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f4911f640 1 --2- 172.21.15.97:0/1325213014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 0x55ebb8e1d1f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c000f80 tx=0x7f2f3c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c2e19e37ccd189 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f39ffb640 1 -- 172.21.15.97:0/1325213014 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00e2f0 con 0x55ebb8e572c0 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f39ffb640 1 -- 172.21.15.97:0/1325213014 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f3c00a870 con 0x55ebb8e572c0 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.962+0000 7f2f39ffb640 1 -- 172.21.15.97:0/1325213014 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00ac60 con 0x55ebb8e572c0 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.963+0000 7f2f4d76a740 1 -- 172.21.15.97:0/1325213014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 msgr2=0x55ebb8e1d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.963+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/1325213014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 0x55ebb8e1d1f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c000f80 tx=0x7f2f3c014000 comp rx=0 tx=0).stop 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.963+0000 7f2f4d76a740 1 -- 172.21.15.97:0/1325213014 shutdown_connections 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.963+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/1325213014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e572c0 0x55ebb8e1d1f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.963+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/1325213014 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4ef00 0x55ebb8e6da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.963+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/1325213014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e4eb20 0x55ebb8e27de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.963+0000 7f2f4d76a740 1 -- 172.21.15.97:0/1325213014 >> 172.21.15.97:0/1325213014 conn(0x55ebb8e22000 msgr2=0x55ebb8e28600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.963+0000 7f2f4d76a740 1 -- 172.21.15.97:0/1325213014 shutdown_connections 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.963+0000 7f2f4d76a740 1 -- 172.21.15.97:0/1325213014 wait complete. 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.963+0000 7f2f4d76a740 1 -- start start 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.964+0000 7f2f4d76a740 1 Processor -- start 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.964+0000 7f2f4d76a740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4eb20 0x55ebb8e4b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.964+0000 7f2f4d76a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4ef00 0x55ebb8e4b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.964+0000 7f2f4d76a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e572c0 0x55ebb8eec380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.964+0000 7f2f4d76a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ebb8e14910 con 0x55ebb8e4ef00 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.964+0000 7f2f4d76a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441f5f30 con 0x55ebb8e572c0 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.964+0000 7f2f4d76a740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55ebb8e1d730 con 0x55ebb8e4eb20 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.964+0000 7f2f4d76a740 10 client.0 _ll_register_callbacks cb 0x7f2f3c007760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.964+0000 7f2f43fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4ef00 0x55ebb8e4b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.964+0000 7f2f43fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4ef00 0x55ebb8e4b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59860/0 (socket says 172.21.15.97:59860) 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.964+0000 7f2f43fff640 1 -- 172.21.15.97:0/3575379364 learned_addr learned my addr 172.21.15.97:0/3575379364 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:52.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.964+0000 7f2f4891e640 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4eb20 0x55ebb8e4b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.964+0000 7f2f4911f640 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e572c0 0x55ebb8eec380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f43fff640 1 -- 172.21.15.97:0/3575379364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4eb20 msgr2=0x55ebb8e4b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f43fff640 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4eb20 0x55ebb8e4b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f43fff640 1 -- 172.21.15.97:0/3575379364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e572c0 msgr2=0x55ebb8eec380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f43fff640 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e572c0 0x55ebb8eec380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f43fff640 1 -- 172.21.15.97:0/3575379364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebb8e570b0 con 0x55ebb8e4ef00 2024-03-29T04:08:52.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f4891e640 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4eb20 0x55ebb8e4b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:52.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f43fff640 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4ef00 0x55ebb8e4b950 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c055ac0 tx=0x7f2f2c059080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d0767c12e570d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f237fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-29T04:08:52.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f237fe640 1 -- 172.21.15.97:0/3575379364 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c087070 con 0x55ebb8e4ef00 2024-03-29T04:08:52.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f237fe640 1 -- 172.21.15.97:0/3575379364 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2f2c058cb0 con 0x55ebb8e4ef00 2024-03-29T04:08:52.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f4d76a740 1 -- 172.21.15.97:0/3575379364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f3c00a870 con 0x55ebb8e4ef00 2024-03-29T04:08:52.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f237fe640 1 -- 172.21.15.97:0/3575379364 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f2f2c0622d0 con 0x55ebb8e4ef00 2024-03-29T04:08:52.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f4d76a740 10 client.5316 fetch_fsmap learned FSMap version 16 2024-03-29T04:08:52.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f4d76a740 1 -- 172.21.15.97:0/3575379364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55ebb8e7b3e0 con 0x55ebb8e4ef00 2024-03-29T04:08:52.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f237fe640 1 -- 172.21.15.97:0/3575379364 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f3c00a870 con 0x55ebb8e4ef00 2024-03-29T04:08:52.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f237fe640 1 -- 172.21.15.97:0/3575379364 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f2f2c071ce0 con 0x55ebb8e4ef00 2024-03-29T04:08:52.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f4d76a740 10 client.5316 fetch_fsmap finished waiting for FSMap version 16 2024-03-29T04:08:52.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f4d76a740 10 client.5316 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-29T04:08:52.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f4d76a740 20 client.5316 populate_metadata read hostname 'smithi097' 2024-03-29T04:08:52.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ebb8e52af0 0x55ebb8eec8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f4d76a740 4 client.5316 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-03-29T04:08:52.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f4d76a740 1 -- 172.21.15.97:0/3575379364 --> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55ebb8d92ca0 con 0x55ebb8e52af0 2024-03-29T04:08:52.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.965+0000 7f2f4891e640 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ebb8e52af0 0x55ebb8eec8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.966+0000 7f2f4891e640 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ebb8e52af0 0x55ebb8eec8c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b34d707e78ef6 server_cookie=18c0d826d65acb51 in_seq=0 out_seq=0 2024-03-29T04:08:52.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.966+0000 7f2f237fe640 10 client.5316 ms_handle_connect on v2:172.21.15.97:6834/3142898446 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.967+0000 7f2f237fe640 1 -- 172.21.15.97:0/3575379364 <== mds.0 v2:172.21.15.97:6834/3142898446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ebb8d92ca0 con 0x55ebb8e52af0 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.967+0000 7f2f237fe640 10 client.5316 handle_command_reply: tid=0 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.967+0000 7f2f4d76a740 1 client.5316 shutdown 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.968+0000 7f2f4d76a740 1 -- 172.21.15.97:0/3575379364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4ef00 msgr2=0x55ebb8e4b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.968+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4ef00 0x55ebb8e4b950 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c055ac0 tx=0x7f2f2c059080 comp rx=0 tx=0).stop 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.968+0000 7f2f4d76a740 1 -- 172.21.15.97:0/3575379364 shutdown_connections 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.968+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ebb8e52af0 0x55ebb8eec8c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.968+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ebb8e572c0 0x55ebb8eec380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.968+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebb8e4ef00 0x55ebb8e4b950 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.968+0000 7f2f4d76a740 1 --2- 172.21.15.97:0/3575379364 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ebb8e4eb20 0x55ebb8e4b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.968+0000 7f2f4d76a740 1 -- 172.21.15.97:0/3575379364 >> 172.21.15.97:0/3575379364 conn(0x55ebb8e22000 msgr2=0x55ebb8e512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.968+0000 7f2f237fe640 0 client.5316 ms_handle_reset on v2:172.21.15.97:6834/3142898446 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.968+0000 7f2f4d76a740 1 -- 172.21.15.97:0/3575379364 shutdown_connections 2024-03-29T04:08:52.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.968+0000 7f2f4d76a740 1 -- 172.21.15.97:0/3575379364 wait complete. 2024-03-29T04:08:52.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.968+0000 7f2f4d76a740 20 client.5316 trim_cache size 0 max 16384 2024-03-29T04:08:52.957 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:08:52.957 INFO:teuthology.orchestra.run.smithi097.stdout: "return_code": 0, 2024-03-29T04:08:52.957 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_tag": "d8075f69-75a5-4242-b3a2-65a930b73c50", 2024-03-29T04:08:52.957 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": "asynchronous" 2024-03-29T04:08:52.957 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:08:52.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.969+0000 7f2f4ab82640 1 -- 172.21.15.97:0/3735137388 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2f2406c530 msgr2=0x7f2f2406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.969+0000 7f2f4ab82640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2f2406c530 0x7f2f2406e9f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2f34036000 tx=0x7f2f3403d040 comp rx=0 tx=0).stop 2024-03-29T04:08:52.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.969+0000 7f2f4ab82640 1 -- 172.21.15.97:0/3735137388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f440d0490 msgr2=0x7f2f4409dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.969+0000 7f2f4ab82640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f440d0490 0x7f2f4409dd10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00c820 tx=0x7f2f2c00cce0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.969+0000 7f2f4ab82640 1 -- 172.21.15.97:0/3735137388 shutdown_connections 2024-03-29T04:08:52.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.969+0000 7f2f4ab82640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2f2406c530 0x7f2f2406e9f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.969+0000 7f2f4ab82640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44156ba0 0x7f2f441536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.969+0000 7f2f4ab82640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f440d0490 0x7f2f4409dd10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.969+0000 7f2f4ab82640 1 --2- 172.21.15.97:0/3735137388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4406d860 0x7f2f4414e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.969+0000 7f2f4ab82640 1 -- 172.21.15.97:0/3735137388 >> 172.21.15.97:0/3735137388 conn(0x7f2f4406b6f0 msgr2=0x7f2f440b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.969+0000 7f2f4ab82640 1 -- 172.21.15.97:0/3735137388 shutdown_connections 2024-03-29T04:08:52.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:52.969+0000 7f2f4ab82640 1 -- 172.21.15.97:0/3735137388 wait complete. 2024-03-29T04:08:52.963 DEBUG:tasks.cephfs.test_uninlining:scrub start response: {'return_code': 0, 'scrub_tag': 'd8075f69-75a5-4242-b3a2-65a930b73c50', 'mode': 'asynchronous'} 2024-03-29T04:08:52.963 DEBUG:tasks.cephfs.test_uninlining:waiting for scrub to complete 2024-03-29T04:08:52.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:08:52.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-29T04:08:53.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.133+0000 7f03b620c640 1 Processor -- start 2024-03-29T04:08:53.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.133+0000 7f03b620c640 1 -- start start 2024-03-29T04:08:53.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.134+0000 7f03b620c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f03b015b3d0 0x7f03b006f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.134+0000 7f03b620c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b0071f20 con 0x7f03b00d0410 2024-03-29T04:08:53.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.134+0000 7f03b620c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b0072200 con 0x7f03b0071860 2024-03-29T04:08:53.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.134+0000 7f03b620c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f03b0071ce0 con 0x7f03b015b3d0 2024-03-29T04:08:53.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.134+0000 7f03aeffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f03b0071860 0x7f03b00b3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:33292/0 (socket says 172.21.15.97:33292) 2024-03-29T04:08:53.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.134+0000 7f03aeffd640 1 -- 172.21.15.97:0/2232785794 learned_addr learned my addr 172.21.15.97:0/2232785794 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:08:53.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.134+0000 7f03af7fe640 1 --2- 172.21.15.97:0/2232785794 >> v2:172.21.15.97:3300/0 conn(0x7f03b015b3d0 0x7f03b006f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.134+0000 7f03ae7fc640 1 -- 172.21.15.97:0/2232785794 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 95076020 0 0) 0x7f03b0072200 con 0x7f03b0071860 2024-03-29T04:08:53.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.134+0000 7f03ae7fc640 1 -- 172.21.15.97:0/2232785794 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f039c003520 con 0x7f03b0071860 2024-03-29T04:08:53.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.134+0000 7f03af7fe640 1 -- 172.21.15.97:0/2232785794 >> v1:172.21.15.154:6789/0 conn(0x7f03b0071860 legacy=0x7f03b00b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.135+0000 7f03af7fe640 1 -- 172.21.15.97:0/2232785794 >> v1:172.21.15.97:6789/0 conn(0x7f03b00d0410 legacy=0x7f03b00adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.135+0000 7f03ae7fc640 1 -- 172.21.15.97:0/2232785794 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4124540506 0 0) 0x7f03b0071f20 con 0x7f03b00d0410 2024-03-29T04:08:53.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.135+0000 7f03af7fe640 1 -- 172.21.15.97:0/2232785794 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b00724e0 con 0x7f03b015b3d0 2024-03-29T04:08:53.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.136+0000 7f03af7fe640 1 --2- 172.21.15.97:0/2232785794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b015b3d0 0x7f03b006f320 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f03a0014340 tx=0x7f03a0034a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b904e94302b10e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.136+0000 7f03ae7fc640 1 -- 172.21.15.97:0/2232785794 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0038840 con 0x7f03b015b3d0 2024-03-29T04:08:53.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.136+0000 7f03ae7fc640 1 -- 172.21.15.97:0/2232785794 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a00389e0 con 0x7f03b015b3d0 2024-03-29T04:08:53.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.136+0000 7f03ae7fc640 1 -- 172.21.15.97:0/2232785794 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0038d90 con 0x7f03b015b3d0 2024-03-29T04:08:53.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.137+0000 7f03b620c640 1 -- 172.21.15.97:0/2232785794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b015b3d0 msgr2=0x7f03b006f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.137+0000 7f03b620c640 1 --2- 172.21.15.97:0/2232785794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b015b3d0 0x7f03b006f320 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f03a0014340 tx=0x7f03a0034a20 comp rx=0 tx=0).stop 2024-03-29T04:08:53.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.137+0000 7f03b620c640 1 -- 172.21.15.97:0/2232785794 shutdown_connections 2024-03-29T04:08:53.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.137+0000 7f03b620c640 1 --2- 172.21.15.97:0/2232785794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b015b3d0 0x7f03b006f320 secure :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f03a0014340 tx=0x7f03a0034a20 comp rx=0 tx=0).stop 2024-03-29T04:08:53.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.137+0000 7f03b620c640 1 -- 172.21.15.97:0/2232785794 >> 172.21.15.97:0/2232785794 conn(0x7f03b006b6f0 msgr2=0x7f03b006bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.137+0000 7f03b620c640 1 -- 172.21.15.97:0/2232785794 shutdown_connections 2024-03-29T04:08:53.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.137+0000 7f03b620c640 1 -- 172.21.15.97:0/2232785794 wait complete. 2024-03-29T04:08:53.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.139+0000 7f03b620c640 1 Processor -- start 2024-03-29T04:08:53.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.139+0000 7f03b620c640 1 -- start start 2024-03-29T04:08:53.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.140+0000 7f03b620c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b0071860 0x7f03b00998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.140+0000 7f03b620c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03b00d0410 0x7f03b015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.140+0000 7f03b620c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b015bf00 0x7f03b00b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.141+0000 7f03b620c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03b00b3f70 con 0x7f03b0071860 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.141+0000 7f03b620c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03b015b7b0 con 0x7f03b015bf00 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.141+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03b00d0410 0x7f03b015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.141+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03b00d0410 0x7f03b015b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:48512/0 (socket says 172.21.15.97:48512) 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.141+0000 7f03aeffd640 1 -- 172.21.15.97:0/1016073511 learned_addr learned my addr 172.21.15.97:0/1016073511 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.141+0000 7f03af7fe640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b0071860 0x7f03b00998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.141+0000 7f03affff640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b015bf00 0x7f03b00b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.141+0000 7f03b620c640 1 -- 172.21.15.97:0/1016073511 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f03b00b40f0 con 0x7f03b00d0410 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.141+0000 7f03aeffd640 1 -- 172.21.15.97:0/1016073511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b015bf00 msgr2=0x7f03b00b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.141+0000 7f03aeffd640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b015bf00 0x7f03b00b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.141+0000 7f03aeffd640 1 -- 172.21.15.97:0/1016073511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b0071860 msgr2=0x7f03b00998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.141+0000 7f03aeffd640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b0071860 0x7f03b00998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.141+0000 7f03aeffd640 1 -- 172.21.15.97:0/1016073511 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b015c480 con 0x7f03b00d0410 2024-03-29T04:08:53.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.142+0000 7f03aeffd640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03b00d0410 0x7f03b015b9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f039800c460 tx=0x7f039800c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.142+0000 7f03a7fff640 1 -- 172.21.15.97:0/1016073511 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398017070 con 0x7f03b00d0410 2024-03-29T04:08:53.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.142+0000 7f03a7fff640 1 -- 172.21.15.97:0/1016073511 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0398005ce0 con 0x7f03b00d0410 2024-03-29T04:08:53.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.142+0000 7f03a7fff640 1 -- 172.21.15.97:0/1016073511 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398007290 con 0x7f03b00d0410 2024-03-29T04:08:53.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.142+0000 7f03b620c640 1 -- 172.21.15.97:0/1016073511 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b015c760 con 0x7f03b00d0410 2024-03-29T04:08:53.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.142+0000 7f03b620c640 1 -- 172.21.15.97:0/1016073511 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b015cd20 con 0x7f03b00d0410 2024-03-29T04:08:53.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.145+0000 7f03a7fff640 1 -- 172.21.15.97:0/1016073511 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f039801a070 con 0x7f03b00d0410 2024-03-29T04:08:53.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.145+0000 7f03a7fff640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f039006c530 0x7f039006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.145+0000 7f03af7fe640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f039006c530 0x7f039006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.146+0000 7f03a7fff640 1 -- 172.21.15.97:0/1016073511 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f039808a9b0 con 0x7f03b00d0410 2024-03-29T04:08:53.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.146+0000 7f03b8df1740 1 Processor -- start 2024-03-29T04:08:53.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.146+0000 7f03b8df1740 1 -- start start 2024-03-29T04:08:53.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.146+0000 7f03af7fe640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f039006c530 0x7f039006e9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f03a003f000 tx=0x7f03a0044040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.147+0000 7f03b8df1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 0x55ca5e8a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.147+0000 7f03b8df1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e8d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.147+0000 7f03b8df1740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8a26e0 0x55ca5e8d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.147+0000 7f03b8df1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ca5e80c880 con 0x55ca5e8cf200 2024-03-29T04:08:53.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.147+0000 7f03b8df1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ca5e7b48f0 con 0x55ca5e8cf5e0 2024-03-29T04:08:53.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.147+0000 7f03b8df1740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55ca5e808830 con 0x55ca5e8a26e0 2024-03-29T04:08:53.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.147+0000 7f03af7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 0x55ca5e8a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.147+0000 7f03af7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 0x55ca5e8a21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59888/0 (socket says 172.21.15.97:59888) 2024-03-29T04:08:53.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.147+0000 7f03af7fe640 1 -- 172.21.15.97:0/1549275854 learned_addr learned my addr 172.21.15.97:0/1549275854 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:53.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.147+0000 7f03aeffd640 1 --2- 172.21.15.97:0/1549275854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e8d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.148+0000 7f03af7fe640 1 -- 172.21.15.97:0/1549275854 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8a26e0 msgr2=0x55ca5e8d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:53.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.148+0000 7f03af7fe640 1 --2- 172.21.15.97:0/1549275854 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8a26e0 0x55ca5e8d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.148+0000 7f03af7fe640 1 -- 172.21.15.97:0/1549275854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 msgr2=0x55ca5e8d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:53.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.148+0000 7f03af7fe640 1 --2- 172.21.15.97:0/1549275854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e8d1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.148+0000 7f03af7fe640 1 -- 172.21.15.97:0/1549275854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca5e7bf140 con 0x55ca5e8cf200 2024-03-29T04:08:53.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.148+0000 7f03af7fe640 1 --2- 172.21.15.97:0/1549275854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 0x55ca5e8a21a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f03a0035030 tx=0x7f03a0039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3062c21905df9b28 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.148+0000 7f03a57fa640 1 -- 172.21.15.97:0/1549275854 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a000f620 con 0x55ca5e8cf200 2024-03-29T04:08:53.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.148+0000 7f03a57fa640 1 -- 172.21.15.97:0/1549275854 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a0003610 con 0x55ca5e8cf200 2024-03-29T04:08:53.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.149+0000 7f03b8df1740 1 -- 172.21.15.97:0/1549275854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 msgr2=0x55ca5e8a21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.149+0000 7f03b8df1740 1 --2- 172.21.15.97:0/1549275854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 0x55ca5e8a21a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f03a0035030 tx=0x7f03a0039000 comp rx=0 tx=0).stop 2024-03-29T04:08:53.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.149+0000 7f03b8df1740 1 -- 172.21.15.97:0/1549275854 shutdown_connections 2024-03-29T04:08:53.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.149+0000 7f03b8df1740 1 --2- 172.21.15.97:0/1549275854 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8a26e0 0x55ca5e8d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.149+0000 7f03b8df1740 1 --2- 172.21.15.97:0/1549275854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e8d1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.149+0000 7f03b8df1740 1 --2- 172.21.15.97:0/1549275854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 0x55ca5e8a21a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.150+0000 7f03b8df1740 1 -- 172.21.15.97:0/1549275854 >> 172.21.15.97:0/1549275854 conn(0x55ca5e8a4410 msgr2=0x55ca5e8ecf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.150+0000 7f03b8df1740 1 -- 172.21.15.97:0/1549275854 shutdown_connections 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.150+0000 7f03b8df1740 1 -- 172.21.15.97:0/1549275854 wait complete. 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.150+0000 7f03b8df1740 1 -- start start 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.150+0000 7f03b8df1740 1 Processor -- start 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.151+0000 7f03b8df1740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8cf200 0x55ca5e901270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.151+0000 7f03b8df1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e8d55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.151+0000 7f03b8df1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8c77c0 0x55ca5e8d5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.151+0000 7f03b8df1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ca5e8a6ab0 con 0x55ca5e8cf5e0 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.151+0000 7f03b8df1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ca5e7c7cf0 con 0x55ca5e8c77c0 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.151+0000 7f03b8df1740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55ca5e8a92f0 con 0x55ca5e8cf200 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.151+0000 7f03b8df1740 10 client.0 _ll_register_callbacks cb 0x55ca5e7b9f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.151+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e8d55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.151+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e8d55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59896/0 (socket says 172.21.15.97:59896) 2024-03-29T04:08:53.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.151+0000 7f03aeffd640 1 -- 172.21.15.97:0/2689037687 learned_addr learned my addr 172.21.15.97:0/2689037687 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:53.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.151+0000 7f03affff640 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8c77c0 0x55ca5e8d5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.151+0000 7f03af7fe640 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8cf200 0x55ca5e901270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.152+0000 7f03affff640 1 -- 172.21.15.97:0/2689037687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8cf200 msgr2=0x55ca5e901270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:53.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.152+0000 7f03affff640 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8cf200 0x55ca5e901270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.152+0000 7f03affff640 1 -- 172.21.15.97:0/2689037687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf5e0 msgr2=0x55ca5e8d55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:53.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.152+0000 7f03affff640 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e8d55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.152+0000 7f03affff640 1 -- 172.21.15.97:0/2689037687 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca5e5ea7d0 con 0x55ca5e8c77c0 2024-03-29T04:08:53.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.152+0000 7f03aeffd640 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e8d55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:53.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.152+0000 7f03affff640 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8c77c0 0x55ca5e8d5b10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f03a8007fd0 tx=0x7f03a800fa20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3c197efe6b9357c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.152+0000 7f038e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-29T04:08:53.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.152+0000 7f038e7fc640 1 -- 172.21.15.97:0/2689037687 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8003b80 con 0x55ca5e8c77c0 2024-03-29T04:08:53.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.152+0000 7f03b8df1740 1 -- 172.21.15.97:0/2689037687 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03a0003610 con 0x55ca5e8c77c0 2024-03-29T04:08:53.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.152+0000 7f038e7fc640 1 -- 172.21.15.97:0/2689037687 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f03a8003d20 con 0x55ca5e8c77c0 2024-03-29T04:08:53.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.152+0000 7f038e7fc640 1 -- 172.21.15.97:0/2689037687 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f03a8005070 con 0x55ca5e8c77c0 2024-03-29T04:08:53.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.152+0000 7f03b8df1740 10 client.5248 fetch_fsmap learned FSMap version 16 2024-03-29T04:08:53.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.153+0000 7f03b8df1740 1 -- 172.21.15.97:0/2689037687 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55ca5e5203f0 con 0x55ca5e8c77c0 2024-03-29T04:08:53.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.153+0000 7f038e7fc640 1 -- 172.21.15.97:0/2689037687 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a0003610 con 0x55ca5e8c77c0 2024-03-29T04:08:53.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.153+0000 7f038e7fc640 1 -- 172.21.15.97:0/2689037687 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f03a8005210 con 0x55ca5e8c77c0 2024-03-29T04:08:53.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.153+0000 7f03b8df1740 10 client.5248 fetch_fsmap finished waiting for FSMap version 16 2024-03-29T04:08:53.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.153+0000 7f03b8df1740 10 client.5248 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-29T04:08:53.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.153+0000 7f03b8df1740 20 client.5248 populate_metadata read hostname 'smithi097' 2024-03-29T04:08:53.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.153+0000 7f03b8df1740 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ca5e8d0c50 0x55ca5e8d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.153+0000 7f03b8df1740 4 client.5248 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:08:53.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.153+0000 7f03b8df1740 1 -- 172.21.15.97:0/2689037687 --> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca5e819dc0 con 0x55ca5e8d0c50 2024-03-29T04:08:53.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.153+0000 7f03af7fe640 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ca5e8d0c50 0x55ca5e8d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.154+0000 7f038e7fc640 10 client.5248 ms_handle_connect on v2:172.21.15.97:6834/3142898446 2024-03-29T04:08:53.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.154+0000 7f03af7fe640 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ca5e8d0c50 0x55ca5e8d10c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55a5fa17b0f5f164 server_cookie=f3a26ab40990c013 in_seq=0 out_seq=0 2024-03-29T04:08:53.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.156+0000 7f038e7fc640 1 -- 172.21.15.97:0/2689037687 <== mds.0 v2:172.21.15.97:6834/3142898446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55ca5e819dc0 con 0x55ca5e8d0c50 2024-03-29T04:08:53.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.156+0000 7f038e7fc640 10 client.5248 handle_command_reply: tid=0 2024-03-29T04:08:53.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.156+0000 7f03b8df1740 1 client.5248 shutdown 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.157+0000 7f03b8df1740 1 -- 172.21.15.97:0/2689037687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8c77c0 msgr2=0x55ca5e8d5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.157+0000 7f03b8df1740 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8c77c0 0x55ca5e8d5b10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f03a8007fd0 tx=0x7f03a800fa20 comp rx=0 tx=0).stop 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.157+0000 7f03b8df1740 1 -- 172.21.15.97:0/2689037687 shutdown_connections 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.157+0000 7f03b8df1740 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ca5e8d0c50 0x55ca5e8d10c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.157+0000 7f03b8df1740 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8c77c0 0x55ca5e8d5b10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.157+0000 7f03b8df1740 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e8d55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.157+0000 7f03b8df1740 1 --2- 172.21.15.97:0/2689037687 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8cf200 0x55ca5e901270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.157+0000 7f03b8df1740 1 -- 172.21.15.97:0/2689037687 >> 172.21.15.97:0/2689037687 conn(0x55ca5e8a26e0 msgr2=0x55ca5e8a60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.157+0000 7f038e7fc640 0 client.5248 ms_handle_reset on v2:172.21.15.97:6834/3142898446 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.157+0000 7f03b8df1740 1 -- 172.21.15.97:0/2689037687 shutdown_connections 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.157+0000 7f03b8df1740 1 -- 172.21.15.97:0/2689037687 wait complete. 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.157+0000 7f03b8df1740 20 client.5248 trim_cache size 0 max 16384 2024-03-29T04:08:53.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.184+0000 7f03b8df1740 1 Processor -- start 2024-03-29T04:08:53.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.184+0000 7f03b8df1740 1 -- start start 2024-03-29T04:08:53.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.185+0000 7f03b8df1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8c77c0 0x55ca5e8ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.186+0000 7f03af7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8c77c0 0x55ca5e8ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.186+0000 7f03af7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8c77c0 0x55ca5e8ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59908/0 (socket says 172.21.15.97:59908) 2024-03-29T04:08:53.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.186+0000 7f03af7fe640 1 -- 172.21.15.97:0/1014744039 learned_addr learned my addr 172.21.15.97:0/1014744039 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:53.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.186+0000 7f03b8df1740 1 --2- 172.21.15.97:0/1014744039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8cf200 0x55ca5e906e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.186+0000 7f03b8df1740 1 --2- 172.21.15.97:0/1014744039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e9073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.186+0000 7f03b8df1740 1 -- 172.21.15.97:0/1014744039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ca5e8a81a0 con 0x55ca5e8c77c0 2024-03-29T04:08:53.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.186+0000 7f03b8df1740 1 -- 172.21.15.97:0/1014744039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03b01fc8f0 con 0x55ca5e8cf5e0 2024-03-29T04:08:53.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.186+0000 7f03b8df1740 1 -- 172.21.15.97:0/1014744039 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f03b01fb980 con 0x55ca5e8cf200 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.186+0000 7f03affff640 1 --2- 172.21.15.97:0/1014744039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e9073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.186+0000 7f03aeffd640 1 --2- 172.21.15.97:0/1014744039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8cf200 0x55ca5e906e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.186+0000 7f03af7fe640 1 -- 172.21.15.97:0/1014744039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8cf200 msgr2=0x55ca5e906e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.187+0000 7f03af7fe640 1 --2- 172.21.15.97:0/1014744039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8cf200 0x55ca5e906e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.187+0000 7f03af7fe640 1 -- 172.21.15.97:0/1014744039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 msgr2=0x55ca5e9073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.187+0000 7f03af7fe640 1 --2- 172.21.15.97:0/1014744039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e9073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.187+0000 7f03af7fe640 1 -- 172.21.15.97:0/1014744039 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca5e4e00f0 con 0x55ca5e8c77c0 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.187+0000 7f03affff640 1 --2- 172.21.15.97:0/1014744039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e9073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.187+0000 7f03af7fe640 1 --2- 172.21.15.97:0/1014744039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8c77c0 0x55ca5e8ca7e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f03a0035030 tx=0x7f03a000f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398aafde1a167024 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.187+0000 7f03a4ff9640 1 -- 172.21.15.97:0/1014744039 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a006b070 con 0x55ca5e8c77c0 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.187+0000 7f03a4ff9640 1 -- 172.21.15.97:0/1014744039 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a0003b60 con 0x55ca5e8c77c0 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.187+0000 7f03a4ff9640 1 -- 172.21.15.97:0/1014744039 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0041410 con 0x55ca5e8c77c0 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.188+0000 7f03b8df1740 1 -- 172.21.15.97:0/1014744039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8c77c0 msgr2=0x55ca5e8ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.188+0000 7f03b8df1740 1 --2- 172.21.15.97:0/1014744039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8c77c0 0x55ca5e8ca7e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f03a0035030 tx=0x7f03a000f920 comp rx=0 tx=0).stop 2024-03-29T04:08:53.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.188+0000 7f03b8df1740 1 -- 172.21.15.97:0/1014744039 shutdown_connections 2024-03-29T04:08:53.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.188+0000 7f03b8df1740 1 --2- 172.21.15.97:0/1014744039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e9073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.188+0000 7f03b8df1740 1 --2- 172.21.15.97:0/1014744039 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8cf200 0x55ca5e906e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.188+0000 7f03b8df1740 1 --2- 172.21.15.97:0/1014744039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8c77c0 0x55ca5e8ca7e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.188+0000 7f03b8df1740 1 -- 172.21.15.97:0/1014744039 >> 172.21.15.97:0/1014744039 conn(0x55ca5e8a26e0 msgr2=0x55ca5e8a33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.188+0000 7f03b8df1740 1 -- 172.21.15.97:0/1014744039 shutdown_connections 2024-03-29T04:08:53.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.188+0000 7f03b8df1740 1 -- 172.21.15.97:0/1014744039 wait complete. 2024-03-29T04:08:53.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.189+0000 7f03b8df1740 1 -- start start 2024-03-29T04:08:53.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.189+0000 7f03b8df1740 1 Processor -- start 2024-03-29T04:08:53.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.190+0000 7f03b8df1740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8c77c0 0x55ca5e8f04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.190+0000 7f03b8df1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 0x55ca5e8f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.190+0000 7f03b8df1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e967980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.190+0000 7f03b8df1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03b0155610 con 0x55ca5e8cf200 2024-03-29T04:08:53.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.190+0000 7f03b8df1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03b0154fc0 con 0x55ca5e8cf5e0 2024-03-29T04:08:53.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.190+0000 7f03b8df1740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55ca5e968580 con 0x55ca5e8c77c0 2024-03-29T04:08:53.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.190+0000 7f03b8df1740 10 client.0 _ll_register_callbacks cb 0x7f03a8001900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:08:53.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.190+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 0x55ca5e8f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.190+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 0x55ca5e8f09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59910/0 (socket says 172.21.15.97:59910) 2024-03-29T04:08:53.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.190+0000 7f03aeffd640 1 -- 172.21.15.97:0/525805169 learned_addr learned my addr 172.21.15.97:0/525805169 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.190+0000 7f03affff640 1 --2- 172.21.15.97:0/525805169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e967980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.191+0000 7f03aeffd640 1 -- 172.21.15.97:0/525805169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8c77c0 msgr2=0x55ca5e8f04a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.191+0000 7f03aeffd640 1 --2- 172.21.15.97:0/525805169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8c77c0 0x55ca5e8f04a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.191+0000 7f03aeffd640 1 -- 172.21.15.97:0/525805169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 msgr2=0x55ca5e967980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.191+0000 7f03aeffd640 1 --2- 172.21.15.97:0/525805169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e967980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.191+0000 7f03aeffd640 1 -- 172.21.15.97:0/525805169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca5e89dde0 con 0x55ca5e8cf200 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.191+0000 7f03aeffd640 1 --2- 172.21.15.97:0/525805169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 0x55ca5e8f09e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0398088e80 tx=0x7f0398058ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21749c5457dd566 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.191+0000 7f038e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.191+0000 7f038e7fc640 1 -- 172.21.15.97:0/525805169 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398086020 con 0x55ca5e8cf200 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.191+0000 7f038e7fc640 1 -- 172.21.15.97:0/525805169 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f0398007630 con 0x55ca5e8cf200 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.191+0000 7f03b8df1740 1 -- 172.21.15.97:0/525805169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03a0003b60 con 0x55ca5e8cf200 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.191+0000 7f038e7fc640 1 -- 172.21.15.97:0/525805169 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f0398056d40 con 0x55ca5e8cf200 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.191+0000 7f03b8df1740 10 client.5337 fetch_fsmap learned FSMap version 16 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.191+0000 7f03b8df1740 1 -- 172.21.15.97:0/525805169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55ca5e900bc0 con 0x55ca5e8cf200 2024-03-29T04:08:53.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.192+0000 7f038e7fc640 1 -- 172.21.15.97:0/525805169 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a0003b60 con 0x55ca5e8cf200 2024-03-29T04:08:53.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.192+0000 7f038e7fc640 1 -- 172.21.15.97:0/525805169 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f0398057dd0 con 0x55ca5e8cf200 2024-03-29T04:08:53.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.192+0000 7f03b8df1740 10 client.5337 fetch_fsmap finished waiting for FSMap version 16 2024-03-29T04:08:53.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.192+0000 7f03b8df1740 10 client.5337 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-29T04:08:53.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.192+0000 7f03b8df1740 20 client.5337 populate_metadata read hostname 'smithi097' 2024-03-29T04:08:53.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.192+0000 7f03b8df1740 1 --2- 172.21.15.97:0/525805169 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ca5e967ec0 0x55ca5e8c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.192+0000 7f03b8df1740 4 client.5337 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T04:08:53.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.192+0000 7f03b8df1740 1 -- 172.21.15.97:0/525805169 --> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca5e814110 con 0x55ca5e967ec0 2024-03-29T04:08:53.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.192+0000 7f03af7fe640 1 --2- 172.21.15.97:0/525805169 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ca5e967ec0 0x55ca5e8c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.192+0000 7f038e7fc640 10 client.5337 ms_handle_connect on v2:172.21.15.97:6834/3142898446 2024-03-29T04:08:53.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.193+0000 7f03af7fe640 1 --2- 172.21.15.97:0/525805169 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ca5e967ec0 0x55ca5e8c3fa0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8f854516af0ce92 server_cookie=7713047a26cbbd4c in_seq=0 out_seq=0 2024-03-29T04:08:53.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.195+0000 7f038e7fc640 1 -- 172.21.15.97:0/525805169 <== mds.0 v2:172.21.15.97:6834/3142898446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+564 (crc 0 0 0) 0x55ca5e814110 con 0x55ca5e967ec0 2024-03-29T04:08:53.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.195+0000 7f038e7fc640 10 client.5337 handle_command_reply: tid=0 2024-03-29T04:08:53.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.195+0000 7f03b8df1740 1 client.5337 shutdown 2024-03-29T04:08:53.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.195+0000 7f03b8df1740 1 -- 172.21.15.97:0/525805169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 msgr2=0x55ca5e8f09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.195+0000 7f03b8df1740 1 --2- 172.21.15.97:0/525805169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 0x55ca5e8f09e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0398088e80 tx=0x7f0398058ad0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.196+0000 7f03b8df1740 1 -- 172.21.15.97:0/525805169 shutdown_connections 2024-03-29T04:08:53.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.196+0000 7f03b8df1740 1 --2- 172.21.15.97:0/525805169 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55ca5e967ec0 0x55ca5e8c3fa0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.196+0000 7f03b8df1740 1 --2- 172.21.15.97:0/525805169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ca5e8cf5e0 0x55ca5e967980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.196+0000 7f03b8df1740 1 --2- 172.21.15.97:0/525805169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ca5e8cf200 0x55ca5e8f09e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.196+0000 7f03b8df1740 1 --2- 172.21.15.97:0/525805169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55ca5e8c77c0 0x55ca5e8f04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.196+0000 7f03b8df1740 1 -- 172.21.15.97:0/525805169 >> 172.21.15.97:0/525805169 conn(0x55ca5e8a26e0 msgr2=0x55ca5e8d55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.196+0000 7f038e7fc640 0 client.5337 ms_handle_reset on v2:172.21.15.97:6834/3142898446 2024-03-29T04:08:53.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.196+0000 7f03b8df1740 1 -- 172.21.15.97:0/525805169 shutdown_connections 2024-03-29T04:08:53.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.196+0000 7f03b8df1740 1 -- 172.21.15.97:0/525805169 wait complete. 2024-03-29T04:08:53.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.196+0000 7f03b8df1740 20 client.5337 trim_cache size 0 max 16384 2024-03-29T04:08:53.179 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:08:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T04:08:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": { 2024-03-29T04:08:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: "d8075f69-75a5-4242-b3a2-65a930b73c50": { 2024-03-29T04:08:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: "tag": "d8075f69-75a5-4242-b3a2-65a930b73c50", 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: "options": "recursive" 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_stats": { 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: "d8075f69-75a5-4242-b3a2-65a930b73c50": { 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: "start_time": "2024-03-29T04:08:52.966093+0000", 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_started": 0, 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_passed": 0, 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_failed": 0, 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_skipped": 0 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:08:53.181 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:08:53.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.197+0000 7f03b620c640 1 -- 172.21.15.97:0/1016073511 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f039006c530 msgr2=0x7f039006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.197+0000 7f03b620c640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f039006c530 0x7f039006e9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f03a003f000 tx=0x7f03a0044040 comp rx=0 tx=0).stop 2024-03-29T04:08:53.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.197+0000 7f03b620c640 1 -- 172.21.15.97:0/1016073511 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03b00d0410 msgr2=0x7f03b015b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.197+0000 7f03b620c640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03b00d0410 0x7f03b015b9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f039800c460 tx=0x7f039800c920 comp rx=0 tx=0).stop 2024-03-29T04:08:53.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.197+0000 7f03b620c640 1 -- 172.21.15.97:0/1016073511 shutdown_connections 2024-03-29T04:08:53.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.197+0000 7f03b620c640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f039006c530 0x7f039006e9f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.197+0000 7f03b620c640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b015bf00 0x7f03b00b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.197+0000 7f03b620c640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f03b00d0410 0x7f03b015b9c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.197+0000 7f03b620c640 1 --2- 172.21.15.97:0/1016073511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b0071860 0x7f03b00998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.197+0000 7f03b620c640 1 -- 172.21.15.97:0/1016073511 >> 172.21.15.97:0/1016073511 conn(0x7f03b006b6f0 msgr2=0x7f03b006bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.197+0000 7f03b620c640 1 -- 172.21.15.97:0/1016073511 shutdown_connections 2024-03-29T04:08:53.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:08:53.197+0000 7f03b620c640 1 -- 172.21.15.97:0/1016073511 wait complete. 2024-03-29T04:08:53.189 INFO:tasks.cephfs.filesystem:scrub status for tag:d8075f69-75a5-4242-b3a2-65a930b73c50 - {'path': '/', 'tag': 'd8075f69-75a5-4242-b3a2-65a930b73c50', 'options': 'recursive'} 2024-03-29T04:09:11.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:11.184 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:23.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:23.190 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-29T04:09:23.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.314+0000 7f8d6b6a7640 1 Processor -- start 2024-03-29T04:09:23.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.314+0000 7f8d6b6a7640 1 -- start start 2024-03-29T04:09:23.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d6b6a7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d640d16a0 0x7f8d6406c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d6b6a7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6415ed00 con 0x7f8d6415e920 2024-03-29T04:09:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d6b6a7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d640d21e0 con 0x7f8d640d3280 2024-03-29T04:09:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d6b6a7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8d640d24c0 con 0x7f8d640d16a0 2024-03-29T04:09:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d69443640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8d6415e920 0x7f8d640cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51312/0 (socket says 172.21.15.97:51312) 2024-03-29T04:09:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d69443640 1 -- 172.21.15.97:0/820124077 learned_addr learned my addr 172.21.15.97:0/820124077 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d69c44640 1 --2- 172.21.15.97:0/820124077 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d640d16a0 0x7f8d6406c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2789852401 0 0) 0x7f8d6415ed00 con 0x7f8d6415e920 2024-03-29T04:09:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d50003520 con 0x7f8d6415e920 2024-03-29T04:09:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4051663814 0 0) 0x7f8d50003520 con 0x7f8d6415e920 2024-03-29T04:09:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d640d16a0 msgr2=0x7f8d6406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d5bfff640 1 --2- 172.21.15.97:0/820124077 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d640d16a0 0x7f8d6406c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 >> v1:172.21.15.154:6789/0 conn(0x7f8d640d3280 legacy=0x7f8d640d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.315+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d640d27a0 con 0x7f8d6415e920 2024-03-29T04:09:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f8d60001670 con 0x7f8d6415e920 2024-03-29T04:09:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 >> v1:172.21.15.97:6789/0 conn(0x7f8d6415e920 legacy=0x7f8d640cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d5bfff640 1 --2- 172.21.15.97:0/820124077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d500039e0 0x7f8d50003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d5bfff640 1 --2- 172.21.15.97:0/820124077 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d640d16a0 0x7f8d50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d5bfff640 1 --2- 172.21.15.97:0/820124077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d50004d40 0x7f8d500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d640d24c0 con 0x7f8d50004d40 2024-03-29T04:09:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5000b9f0 con 0x7f8d500039e0 2024-03-29T04:09:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8d5000bbf0 con 0x7f8d640d16a0 2024-03-29T04:09:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d69c44640 1 --2- 172.21.15.97:0/820124077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d50004d40 0x7f8d500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d69443640 1 --2- 172.21.15.97:0/820124077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d500039e0 0x7f8d50003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d68c42640 1 --2- 172.21.15.97:0/820124077 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d640d16a0 0x7f8d50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d69c44640 1 -- 172.21.15.97:0/820124077 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d640d16a0 msgr2=0x7f8d50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d69c44640 1 --2- 172.21.15.97:0/820124077 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d640d16a0 0x7f8d50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d69c44640 1 -- 172.21.15.97:0/820124077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d500039e0 msgr2=0x7f8d50003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d69c44640 1 --2- 172.21.15.97:0/820124077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d500039e0 0x7f8d50003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.316+0000 7f8d69c44640 1 -- 172.21.15.97:0/820124077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d50003520 con 0x7f8d50004d40 2024-03-29T04:09:23.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.317+0000 7f8d69c44640 1 --2- 172.21.15.97:0/820124077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d50004d40 0x7f8d500051c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009fa0 tx=0x7f8d50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a2169b6a6f30e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.319+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c039070 con 0x7f8d50004d40 2024-03-29T04:09:23.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.319+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d5c02ccb0 con 0x7f8d50004d40 2024-03-29T04:09:23.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.319+0000 7f8d5bfff640 1 -- 172.21.15.97:0/820124077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c009bf0 con 0x7f8d50004d40 2024-03-29T04:09:23.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.320+0000 7f8d6b6a7640 1 -- 172.21.15.97:0/820124077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d50004d40 msgr2=0x7f8d500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.320+0000 7f8d6b6a7640 1 --2- 172.21.15.97:0/820124077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d50004d40 0x7f8d500051c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009fa0 tx=0x7f8d50005830 comp rx=0 tx=0).stop 2024-03-29T04:09:23.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.320+0000 7f8d69c44640 1 -- 172.21.15.97:0/820124077 reap_dead start 2024-03-29T04:09:23.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.320+0000 7f8d6b6a7640 1 -- 172.21.15.97:0/820124077 shutdown_connections 2024-03-29T04:09:23.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.320+0000 7f8d6b6a7640 1 -- 172.21.15.97:0/820124077 >> 172.21.15.97:0/820124077 conn(0x7f8d6408bfc0 msgr2=0x7f8d640d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.320+0000 7f8d6b6a7640 1 -- 172.21.15.97:0/820124077 shutdown_connections 2024-03-29T04:09:23.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.320+0000 7f8d6b6a7640 1 -- 172.21.15.97:0/820124077 wait complete. 2024-03-29T04:09:23.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.322+0000 7f8d6b6a7640 1 Processor -- start 2024-03-29T04:09:23.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.322+0000 7f8d6b6a7640 1 -- start start 2024-03-29T04:09:23.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d6b6a7640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d50004d40 0x7f8d640b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d6b6a7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d64158ce0 0x7f8d64159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d6b6a7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d641e8490 0x7f8d641e8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d6b6a7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d640d4f50 con 0x7f8d64158ce0 2024-03-29T04:09:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d6b6a7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8d640d2d60 con 0x7f8d641e8490 2024-03-29T04:09:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d6b6a7640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8d640d36e0 con 0x7f8d50004d40 2024-03-29T04:09:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d68c42640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d64158ce0 0x7f8d64159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d68c42640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d64158ce0 0x7f8d64159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42262/0 (socket says 172.21.15.97:42262) 2024-03-29T04:09:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d68c42640 1 -- 172.21.15.97:0/3807404061 learned_addr learned my addr 172.21.15.97:0/3807404061 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d68c42640 1 -- 172.21.15.97:0/3807404061 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d50004d40 msgr2=0x7f8d640b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d68c42640 1 --2- 172.21.15.97:0/3807404061 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d50004d40 0x7f8d640b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d68c42640 1 -- 172.21.15.97:0/3807404061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d641e8490 msgr2=0x7f8d641e8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d68c42640 1 --2- 172.21.15.97:0/3807404061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d641e8490 0x7f8d641e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.323+0000 7f8d68c42640 1 -- 172.21.15.97:0/3807404061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6415f6e0 con 0x7f8d64158ce0 2024-03-29T04:09:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.324+0000 7f8d68c42640 1 --2- 172.21.15.97:0/3807404061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d64158ce0 0x7f8d64159160 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400f7f0 tx=0x7f8d5400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.324+0000 7f8d59ffb640 1 -- 172.21.15.97:0/3807404061 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d540052e0 con 0x7f8d64158ce0 2024-03-29T04:09:23.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.324+0000 7f8d59ffb640 1 -- 172.21.15.97:0/3807404061 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d54018070 con 0x7f8d64158ce0 2024-03-29T04:09:23.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.324+0000 7f8d6b6a7640 1 -- 172.21.15.97:0/3807404061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6415f9c0 con 0x7f8d64158ce0 2024-03-29T04:09:23.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.324+0000 7f8d59ffb640 1 -- 172.21.15.97:0/3807404061 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5400d8f0 con 0x7f8d64158ce0 2024-03-29T04:09:23.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.324+0000 7f8d6b6a7640 1 -- 172.21.15.97:0/3807404061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d6415ff80 con 0x7f8d64158ce0 2024-03-29T04:09:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.325+0000 7f8d59ffb640 1 -- 172.21.15.97:0/3807404061 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8d5400da90 con 0x7f8d64158ce0 2024-03-29T04:09:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.326+0000 7f8d59ffb640 1 --2- 172.21.15.97:0/3807404061 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f8d4806c560 0x7f8d4806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.326+0000 7f8d59ffb640 1 -- 172.21.15.97:0/3807404061 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8d5408b610 con 0x7f8d64158ce0 2024-03-29T04:09:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.326+0000 7f8d69443640 1 --2- 172.21.15.97:0/3807404061 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f8d4806c560 0x7f8d4806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.326+0000 7f8d69443640 1 --2- 172.21.15.97:0/3807404061 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f8d4806c560 0x7f8d4806ea20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8d60000f60 tx=0x7f8d60000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.327+0000 7f8d6e368740 1 Processor -- start 2024-03-29T04:09:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.327+0000 7f8d6e368740 1 -- start start 2024-03-29T04:09:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.327+0000 7f8d6e368740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d099120200 0x55d0990f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.327+0000 7f8d6e368740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d0991205e0 0x55d099122f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.327+0000 7f8d6e368740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d0990f36e0 0x55d0991234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.327+0000 7f8d6e368740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d09905d880 con 0x55d099120200 2024-03-29T04:09:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.327+0000 7f8d6e368740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d0990058f0 con 0x55d0990f36e0 2024-03-29T04:09:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.327+0000 7f8d6e368740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55d099059830 con 0x55d0991205e0 2024-03-29T04:09:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.327+0000 7f8d69443640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d099120200 0x55d0990f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.328+0000 7f8d69443640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d099120200 0x55d0990f31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42270/0 (socket says 172.21.15.97:42270) 2024-03-29T04:09:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.328+0000 7f8d69443640 1 -- 172.21.15.97:0/3152675818 learned_addr learned my addr 172.21.15.97:0/3152675818 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.328+0000 7f8d69443640 1 -- 172.21.15.97:0/3152675818 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d0991205e0 msgr2=0x55d099122f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.328+0000 7f8d69443640 1 --2- 172.21.15.97:0/3152675818 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d0991205e0 0x55d099122f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.328+0000 7f8d69443640 1 -- 172.21.15.97:0/3152675818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d0990f36e0 msgr2=0x55d0991234d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.328+0000 7f8d69c44640 1 --2- 172.21.15.97:0/3152675818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d0990f36e0 0x55d0991234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.328+0000 7f8d69443640 1 --2- 172.21.15.97:0/3152675818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d0990f36e0 0x55d0991234d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.328+0000 7f8d69443640 1 -- 172.21.15.97:0/3152675818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d099010140 con 0x55d099120200 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.328+0000 7f8d69443640 1 --2- 172.21.15.97:0/3152675818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d099120200 0x55d0990f31a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8d600078a0 tx=0x7f8d60013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f2850ee0322ae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.328+0000 7f8d4effd640 1 -- 172.21.15.97:0/3152675818 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60026070 con 0x55d099120200 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.328+0000 7f8d4effd640 1 -- 172.21.15.97:0/3152675818 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d60004c70 con 0x55d099120200 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.328+0000 7f8d4effd640 1 -- 172.21.15.97:0/3152675818 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60012550 con 0x55d099120200 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.329+0000 7f8d6e368740 1 -- 172.21.15.97:0/3152675818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d099120200 msgr2=0x55d0990f31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.329+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3152675818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d099120200 0x55d0990f31a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8d600078a0 tx=0x7f8d60013000 comp rx=0 tx=0).stop 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.329+0000 7f8d6e368740 1 -- 172.21.15.97:0/3152675818 shutdown_connections 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.329+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3152675818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d0990f36e0 0x55d0991234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.329+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3152675818 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d0991205e0 0x55d099122f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.329+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3152675818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d099120200 0x55d0990f31a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.329+0000 7f8d6e368740 1 -- 172.21.15.97:0/3152675818 >> 172.21.15.97:0/3152675818 conn(0x55d0990f5410 msgr2=0x55d09913df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.329+0000 7f8d6e368740 1 -- 172.21.15.97:0/3152675818 shutdown_connections 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.329+0000 7f8d6e368740 1 -- 172.21.15.97:0/3152675818 wait complete. 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.329+0000 7f8d6e368740 1 -- start start 2024-03-29T04:09:23.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.330+0000 7f8d6e368740 1 Processor -- start 2024-03-29T04:09:23.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.330+0000 7f8d6e368740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 0x55d09911f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.330+0000 7f8d6e368740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 0x55d099125720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.330+0000 7f8d6e368740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d09912be40 0x55d099125c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.330+0000 7f8d6e368740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d0990f7ab0 con 0x55d0991205e0 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.330+0000 7f8d6e368740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d099018cf0 con 0x55d099120200 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.330+0000 7f8d6e368740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55d0990fa2f0 con 0x55d09912be40 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d6e368740 10 client.0 _ll_register_callbacks cb 0x55d09900af00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d68c42640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 0x55d099125720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d68c42640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 0x55d099125720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42276/0 (socket says 172.21.15.97:42276) 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d68c42640 1 -- 172.21.15.97:0/1937674725 learned_addr learned my addr 172.21.15.97:0/1937674725 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d69c44640 1 --2- 172.21.15.97:0/1937674725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d09912be40 0x55d099125c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d69443640 1 --2- 172.21.15.97:0/1937674725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 0x55d09911f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d68c42640 1 -- 172.21.15.97:0/1937674725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d09912be40 msgr2=0x55d099125c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d68c42640 1 --2- 172.21.15.97:0/1937674725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d09912be40 0x55d099125c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d68c42640 1 -- 172.21.15.97:0/1937674725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 msgr2=0x55d09911f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d68c42640 1 --2- 172.21.15.97:0/1937674725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 0x55d09911f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d68c42640 1 -- 172.21.15.97:0/1937674725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d098e3b7d0 con 0x55d0991205e0 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d68c42640 1 --2- 172.21.15.97:0/1937674725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 0x55d099125720 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8d54055990 tx=0x7f8d54005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7871c4bb0913e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d43fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d43fff640 1 -- 172.21.15.97:0/1937674725 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54059530 con 0x55d0991205e0 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d43fff640 1 -- 172.21.15.97:0/1937674725 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8d54058980 con 0x55d0991205e0 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d6e368740 1 -- 172.21.15.97:0/1937674725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d60004c70 con 0x55d0991205e0 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d43fff640 1 -- 172.21.15.97:0/1937674725 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f8d5405d8e0 con 0x55d0991205e0 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d6e368740 10 client.5349 fetch_fsmap learned FSMap version 16 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d6e368740 1 -- 172.21.15.97:0/1937674725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55d098d713f0 con 0x55d0991205e0 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.331+0000 7f8d43fff640 1 -- 172.21.15.97:0/1937674725 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d60004c70 con 0x55d0991205e0 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.332+0000 7f8d43fff640 1 -- 172.21.15.97:0/1937674725 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f8d54071b20 con 0x55d0991205e0 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.332+0000 7f8d6e368740 10 client.5349 fetch_fsmap finished waiting for FSMap version 16 2024-03-29T04:09:23.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.332+0000 7f8d6e368740 10 client.5349 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-29T04:09:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.332+0000 7f8d6e368740 20 client.5349 populate_metadata read hostname 'smithi097' 2024-03-29T04:09:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.332+0000 7f8d6e368740 1 --2- 172.21.15.97:0/1937674725 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55d099128dc0 0x55d099129250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.332+0000 7f8d6e368740 4 client.5349 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:09:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.332+0000 7f8d6e368740 1 -- 172.21.15.97:0/1937674725 --> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d09906adc0 con 0x55d099128dc0 2024-03-29T04:09:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.332+0000 7f8d69443640 1 --2- 172.21.15.97:0/1937674725 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55d099128dc0 0x55d099129250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.332+0000 7f8d43fff640 10 client.5349 ms_handle_connect on v2:172.21.15.97:6834/3142898446 2024-03-29T04:09:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.332+0000 7f8d69443640 1 --2- 172.21.15.97:0/1937674725 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55d099128dc0 0x55d099129250 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d63895cae1214bf server_cookie=2576eb4e2ec99846 in_seq=0 out_seq=0 2024-03-29T04:09:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d43fff640 1 -- 172.21.15.97:0/1937674725 <== mds.0 v2:172.21.15.97:6834/3142898446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55d09906adc0 con 0x55d099128dc0 2024-03-29T04:09:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d43fff640 10 client.5349 handle_command_reply: tid=0 2024-03-29T04:09:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d6e368740 1 client.5349 shutdown 2024-03-29T04:09:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d6e368740 1 -- 172.21.15.97:0/1937674725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 msgr2=0x55d099125720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d6e368740 1 --2- 172.21.15.97:0/1937674725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 0x55d099125720 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8d54055990 tx=0x7f8d54005540 comp rx=0 tx=0).stop 2024-03-29T04:09:23.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d6e368740 1 -- 172.21.15.97:0/1937674725 shutdown_connections 2024-03-29T04:09:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d6e368740 1 --2- 172.21.15.97:0/1937674725 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55d099128dc0 0x55d099129250 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d6e368740 1 --2- 172.21.15.97:0/1937674725 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d09912be40 0x55d099125c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d6e368740 1 --2- 172.21.15.97:0/1937674725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 0x55d099125720 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d43fff640 0 client.5349 ms_handle_reset on v2:172.21.15.97:6834/3142898446 2024-03-29T04:09:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d6e368740 1 --2- 172.21.15.97:0/1937674725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 0x55d09911f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d6e368740 1 -- 172.21.15.97:0/1937674725 >> 172.21.15.97:0/1937674725 conn(0x55d0990f36e0 msgr2=0x55d0990f71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d6e368740 1 -- 172.21.15.97:0/1937674725 shutdown_connections 2024-03-29T04:09:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d6e368740 1 -- 172.21.15.97:0/1937674725 wait complete. 2024-03-29T04:09:23.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.333+0000 7f8d6e368740 20 client.5349 trim_cache size 0 max 16384 2024-03-29T04:09:23.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.352+0000 7f8d6e368740 1 Processor -- start 2024-03-29T04:09:23.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.352+0000 7f8d6e368740 1 -- start start 2024-03-29T04:09:23.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.353+0000 7f8d6e368740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 0x55d099141690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.353+0000 7f8d6e368740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d0991205e0 0x55d09911b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.353+0000 7f8d6e368740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d09912be40 0x55d09911bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.353+0000 7f8d6e368740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d642012c0 con 0x55d09912be40 2024-03-29T04:09:23.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.353+0000 7f8d6e368740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8d642002e0 con 0x55d099120200 2024-03-29T04:09:23.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.353+0000 7f8d6e368740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8d64161090 con 0x55d0991205e0 2024-03-29T04:09:23.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.353+0000 7f8d69c44640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d09912be40 0x55d09911bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.353+0000 7f8d69c44640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d09912be40 0x55d09911bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42286/0 (socket says 172.21.15.97:42286) 2024-03-29T04:09:23.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.353+0000 7f8d69c44640 1 -- 172.21.15.97:0/3395469858 learned_addr learned my addr 172.21.15.97:0/3395469858 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:23.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.353+0000 7f8d69c44640 1 -- 172.21.15.97:0/3395469858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d0991205e0 msgr2=0x55d09911b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.353+0000 7f8d68c42640 1 --2- 172.21.15.97:0/3395469858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d0991205e0 0x55d09911b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.353+0000 7f8d69443640 1 --2- 172.21.15.97:0/3395469858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 0x55d099141690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d69c44640 1 --2- 172.21.15.97:0/3395469858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d0991205e0 0x55d09911b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d69c44640 1 -- 172.21.15.97:0/3395469858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 msgr2=0x55d099141690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d69c44640 1 --2- 172.21.15.97:0/3395469858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 0x55d099141690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d69c44640 1 -- 172.21.15.97:0/3395469858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d09911eef0 con 0x55d09912be40 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d68c42640 1 --2- 172.21.15.97:0/3395469858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d0991205e0 0x55d09911b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d69443640 1 --2- 172.21.15.97:0/3395469858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 0x55d099141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d69c44640 1 --2- 172.21.15.97:0/3395469858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d09912be40 0x55d09911bb40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009f40 tx=0x7f8d5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfc1446ff3fe98f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d4e7fc640 1 -- 172.21.15.97:0/3395469858 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c045070 con 0x55d09912be40 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d4e7fc640 1 -- 172.21.15.97:0/3395469858 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d5c03a040 con 0x55d09912be40 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d4e7fc640 1 -- 172.21.15.97:0/3395469858 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c039040 con 0x55d09912be40 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d6e368740 1 -- 172.21.15.97:0/3395469858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d09912be40 msgr2=0x55d09911bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3395469858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d09912be40 0x55d09911bb40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009f40 tx=0x7f8d5c004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d6e368740 1 -- 172.21.15.97:0/3395469858 shutdown_connections 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3395469858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d09912be40 0x55d09911bb40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3395469858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d0991205e0 0x55d09911b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3395469858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 0x55d099141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.354+0000 7f8d6e368740 1 -- 172.21.15.97:0/3395469858 >> 172.21.15.97:0/3395469858 conn(0x55d0990f36e0 msgr2=0x55d09911f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.355+0000 7f8d6e368740 1 -- 172.21.15.97:0/3395469858 shutdown_connections 2024-03-29T04:09:23.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.355+0000 7f8d6e368740 1 -- 172.21.15.97:0/3395469858 wait complete. 2024-03-29T04:09:23.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.355+0000 7f8d6e368740 1 -- start start 2024-03-29T04:09:23.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.355+0000 7f8d6e368740 1 Processor -- start 2024-03-29T04:09:23.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d6e368740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 0x55d09913f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d6e368740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 0x55d099150560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d6e368740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d09912be40 0x55d099115050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d6e368740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d099122f20 con 0x55d0991205e0 2024-03-29T04:09:23.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d6e368740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8d641616e0 con 0x55d099120200 2024-03-29T04:09:23.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d6e368740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x55d099140350 con 0x55d09912be40 2024-03-29T04:09:23.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d6e368740 10 client.0 _ll_register_callbacks cb 0x7f8d540014a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:09:23.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d68c42640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 0x55d099150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d68c42640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 0x55d099150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42302/0 (socket says 172.21.15.97:42302) 2024-03-29T04:09:23.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d68c42640 1 -- 172.21.15.97:0/3749410197 learned_addr learned my addr 172.21.15.97:0/3749410197 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:23.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d69c44640 1 --2- 172.21.15.97:0/3749410197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d09912be40 0x55d099115050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d69443640 1 --2- 172.21.15.97:0/3749410197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 0x55d09913f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d68c42640 1 -- 172.21.15.97:0/3749410197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d09912be40 msgr2=0x55d099115050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d68c42640 1 --2- 172.21.15.97:0/3749410197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d09912be40 0x55d099115050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d68c42640 1 -- 172.21.15.97:0/3749410197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 msgr2=0x55d09913f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d68c42640 1 --2- 172.21.15.97:0/3749410197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 0x55d09913f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.356+0000 7f8d68c42640 1 -- 172.21.15.97:0/3749410197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d098d310f0 con 0x55d0991205e0 2024-03-29T04:09:23.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d68c42640 1 --2- 172.21.15.97:0/3749410197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 0x55d099150560 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8d540594c0 tx=0x7f8d54007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2801999a623b728 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d43fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-03-29T04:09:23.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d43fff640 1 -- 172.21.15.97:0/3749410197 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54078070 con 0x55d0991205e0 2024-03-29T04:09:23.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d43fff640 1 -- 172.21.15.97:0/3749410197 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8d5401eb10 con 0x55d0991205e0 2024-03-29T04:09:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d6e368740 1 -- 172.21.15.97:0/3749410197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d5c03a040 con 0x55d0991205e0 2024-03-29T04:09:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d43fff640 1 -- 172.21.15.97:0/3749410197 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f8d5407d050 con 0x55d0991205e0 2024-03-29T04:09:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d6e368740 10 client.5355 fetch_fsmap learned FSMap version 16 2024-03-29T04:09:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d6e368740 1 -- 172.21.15.97:0/3749410197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55d0991400f0 con 0x55d0991205e0 2024-03-29T04:09:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d43fff640 1 -- 172.21.15.97:0/3749410197 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d5c03a040 con 0x55d0991205e0 2024-03-29T04:09:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d43fff640 1 -- 172.21.15.97:0/3749410197 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f8d54080020 con 0x55d0991205e0 2024-03-29T04:09:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d6e368740 10 client.5355 fetch_fsmap finished waiting for FSMap version 16 2024-03-29T04:09:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d6e368740 10 client.5355 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-03-29T04:09:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d6e368740 20 client.5355 populate_metadata read hostname 'smithi097' 2024-03-29T04:09:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3749410197 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55d0990f2d20 0x55d099115590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d6e368740 4 client.5355 mds_command: new command op to 5067 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T04:09:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d6e368740 1 -- 172.21.15.97:0/3749410197 --> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d09912d150 con 0x55d0990f2d20 2024-03-29T04:09:23.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.357+0000 7f8d69443640 1 --2- 172.21.15.97:0/3749410197 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55d0990f2d20 0x55d099115590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d43fff640 10 client.5355 ms_handle_connect on v2:172.21.15.97:6834/3142898446 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d69443640 1 --2- 172.21.15.97:0/3749410197 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55d0990f2d20 0x55d099115590 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927581dcf76e02a8 server_cookie=99ae7d09366e6a6e in_seq=0 out_seq=0 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d43fff640 1 -- 172.21.15.97:0/3749410197 <== mds.0 v2:172.21.15.97:6834/3142898446 1 ==== command_reply(tid 0: 0 ) ==== 8+0+376 (crc 0 0 0) 0x55d09912d150 con 0x55d0990f2d20 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d43fff640 10 client.5355 handle_command_reply: tid=0 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d6e368740 1 client.5355 shutdown 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d6e368740 1 -- 172.21.15.97:0/3749410197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 msgr2=0x55d099150560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3749410197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 0x55d099150560 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8d540594c0 tx=0x7f8d54007ca0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d6e368740 1 -- 172.21.15.97:0/3749410197 shutdown_connections 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3749410197 >> [v2:172.21.15.97:6834/3142898446,v1:172.21.15.97:6835/3142898446] conn(0x55d0990f2d20 0x55d099115590 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d43fff640 0 client.5355 ms_handle_reset on v2:172.21.15.97:6834/3142898446 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3749410197 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x55d09912be40 0x55d099115050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3749410197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0991205e0 0x55d099150560 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d6e368740 1 --2- 172.21.15.97:0/3749410197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d099120200 0x55d09913f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d6e368740 1 -- 172.21.15.97:0/3749410197 >> 172.21.15.97:0/3749410197 conn(0x55d0990f36e0 msgr2=0x55d0990f2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d6e368740 1 -- 172.21.15.97:0/3749410197 shutdown_connections 2024-03-29T04:09:23.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d6e368740 1 -- 172.21.15.97:0/3749410197 wait complete. 2024-03-29T04:09:23.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.358+0000 7f8d6e368740 20 client.5355 trim_cache size 0 max 16384 2024-03-29T04:09:23.345 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:09:23.345 INFO:teuthology.orchestra.run.smithi097.stdout: "status": "no active scrubs running", 2024-03-29T04:09:23.345 INFO:teuthology.orchestra.run.smithi097.stdout: "scrubs": {}, 2024-03-29T04:09:23.345 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_stats": { 2024-03-29T04:09:23.345 INFO:teuthology.orchestra.run.smithi097.stdout: "d8075f69-75a5-4242-b3a2-65a930b73c50": { 2024-03-29T04:09:23.346 INFO:teuthology.orchestra.run.smithi097.stdout: "start_time": "2024-03-29T04:08:52.966093+0000", 2024-03-29T04:09:23.346 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/", 2024-03-29T04:09:23.346 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_started": 75, 2024-03-29T04:09:23.346 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_passed": 75, 2024-03-29T04:09:23.346 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_failed": 0, 2024-03-29T04:09:23.346 INFO:teuthology.orchestra.run.smithi097.stdout: "uninline_skipped": 0 2024-03-29T04:09:23.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:09:23.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-03-29T04:09:23.346 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:09:23.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.359+0000 7f8d6b6a7640 1 -- 172.21.15.97:0/3807404061 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f8d4806c560 msgr2=0x7f8d4806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.359+0000 7f8d6b6a7640 1 --2- 172.21.15.97:0/3807404061 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f8d4806c560 0x7f8d4806ea20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8d60000f60 tx=0x7f8d60000ed0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.359+0000 7f8d6b6a7640 1 -- 172.21.15.97:0/3807404061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d64158ce0 msgr2=0x7f8d64159160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.359+0000 7f8d6b6a7640 1 --2- 172.21.15.97:0/3807404061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d64158ce0 0x7f8d64159160 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400f7f0 tx=0x7f8d5400fcb0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.359+0000 7f8d6b6a7640 1 -- 172.21.15.97:0/3807404061 shutdown_connections 2024-03-29T04:09:23.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.359+0000 7f8d6b6a7640 1 --2- 172.21.15.97:0/3807404061 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f8d4806c560 0x7f8d4806ea20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.359+0000 7f8d6b6a7640 1 --2- 172.21.15.97:0/3807404061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d641e8490 0x7f8d641e8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.359+0000 7f8d6b6a7640 1 --2- 172.21.15.97:0/3807404061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d64158ce0 0x7f8d64159160 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.359+0000 7f8d6b6a7640 1 --2- 172.21.15.97:0/3807404061 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d50004d40 0x7f8d640b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.359+0000 7f8d6b6a7640 1 -- 172.21.15.97:0/3807404061 >> 172.21.15.97:0/3807404061 conn(0x7f8d6408bfc0 msgr2=0x7f8d6406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.359+0000 7f8d6b6a7640 1 -- 172.21.15.97:0/3807404061 shutdown_connections 2024-03-29T04:09:23.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:23.359+0000 7f8d6b6a7640 1 -- 172.21.15.97:0/3807404061 wait complete. 2024-03-29T04:09:23.349 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T04:09:23.350 DEBUG:tasks.cephfs.test_uninlining:extracting inodes 2024-03-29T04:09:23.350 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Kconfig 2024-03-29T04:09:23.350 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Kconfig 2024-03-29T04:09:23.365 INFO:teuthology.orchestra.run.smithi097.stdout:1099511627780 linux-5.4/fs/ceph/Kconfig 2024-03-29T04:09:23.365 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Makefile 2024-03-29T04:09:23.365 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Makefile 2024-03-29T04:09:23.420 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628281 linux-5.4/fs/ceph/Makefile 2024-03-29T04:09:23.421 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:09:23.421 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:09:23.475 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628287 linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:09:23.476 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/io.h 2024-03-29T04:09:23.476 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/io.h 2024-03-29T04:09:23.531 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628294 linux-5.4/fs/ceph/io.h 2024-03-29T04:09:23.532 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:09:23.532 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:09:23.587 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628296 linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:09:23.587 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Kconfig 2024-03-29T04:09:23.587 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Kconfig 2024-03-29T04:09:23.641 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628308 linux-5.4/fs/ext2/Kconfig 2024-03-29T04:09:23.642 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:09:23.642 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:09:23.697 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628326 linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:09:23.698 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:09:23.698 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:09:23.751 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628324 linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:09:23.752 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Makefile 2024-03-29T04:09:23.752 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Makefile 2024-03-29T04:09:23.807 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628309 linux-5.4/fs/ext2/Makefile 2024-03-29T04:09:23.807 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/symlink.c 2024-03-29T04:09:23.807 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/symlink.c 2024-03-29T04:09:23.861 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628321 linux-5.4/fs/ext2/symlink.c 2024-03-29T04:09:23.862 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/acl.h 2024-03-29T04:09:23.862 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/acl.h 2024-03-29T04:09:23.916 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628311 linux-5.4/fs/ext2/acl.h 2024-03-29T04:09:23.917 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:09:23.917 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:09:23.973 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628325 linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:09:23.973 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr.h 2024-03-29T04:09:23.974 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr.h 2024-03-29T04:09:24.028 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628323 linux-5.4/fs/ext2/xattr.h 2024-03-29T04:09:24.029 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:09:24.029 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:09:24.083 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628328 linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:09:24.084 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:09:24.084 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:09:24.139 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628341 linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:09:24.140 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:09:24.140 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:09:24.194 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628348 linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:09:24.194 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:09:24.194 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:09:24.249 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628331 linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:09:24.250 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:09:24.250 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:09:24.305 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628339 linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:09:24.306 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:09:24.306 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:09:24.361 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628343 linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:09:24.361 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:09:24.362 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:09:24.417 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628344 linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:09:24.417 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Makefile 2024-03-29T04:09:24.417 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Makefile 2024-03-29T04:09:24.472 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628329 linux-5.4/fs/orangefs/Makefile 2024-03-29T04:09:24.472 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:09:24.472 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:09:24.527 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628352 linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:09:24.528 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:09:24.528 DEBUG:teuthology.orchestra.run.smithi097:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:09:24.583 INFO:teuthology.orchestra.run.smithi097.stdout:1099511628334 linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:09:24.583 DEBUG:tasks.cephfs.test_uninlining:getting inline data info 2024-03-29T04:09:24.583 DEBUG:tasks.cephfs.test_uninlining:getting inode info #1 of 23:1099511627780 2024-03-29T04:09:24.583 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511627780 2024-03-29T04:09:24.584 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T04:09:24.584 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:09:24.667 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:09:24.719 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T04:09:24.719 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:09:24.719 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:09:24.719 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:24.735 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:09:24.735 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:24.793 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:24.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.955+0000 7fdccaaba640 1 Processor -- start 2024-03-29T04:09:24.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.955+0000 7fdccaaba640 1 -- start start 2024-03-29T04:09:24.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.955+0000 7fdccaaba640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdcc4070990 0x7fdcc409b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.955+0000 7fdccaaba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc40d05c0 0x7fdcc409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.955+0000 7fdccaaba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc40d0e10 con 0x7fdcc409af70 2024-03-29T04:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.955+0000 7fdccaaba640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc40d10f0 con 0x7fdcc4070990 2024-03-29T04:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.955+0000 7fdccaaba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc40d13d0 con 0x7fdcc40d05c0 2024-03-29T04:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.956+0000 7fdcc8856640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdcc4070990 0x7fdcc409b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.956+0000 7fdcc8856640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdcc4070990 0x7fdcc409b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42308/0 (socket says 172.21.15.97:42308) 2024-03-29T04:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.956+0000 7fdcc8856640 1 -- 172.21.15.97:0/257007296 learned_addr learned my addr 172.21.15.97:0/257007296 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.956+0000 7fdcbbfff640 1 --2- 172.21.15.97:0/257007296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc40d05c0 0x7fdcc409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.956+0000 7fdcc8856640 1 -- 172.21.15.97:0/257007296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc40d05c0 msgr2=0x7fdcc409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.956+0000 7fdcc8856640 1 --2- 172.21.15.97:0/257007296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc40d05c0 0x7fdcc409aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.956+0000 7fdcc8856640 1 -- 172.21.15.97:0/257007296 >> v1:172.21.15.154:6789/0 conn(0x7fdcc409af70 legacy=0x7fdcc415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.956+0000 7fdcc8856640 1 -- 172.21.15.97:0/257007296 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc40d16b0 con 0x7fdcc4070990 2024-03-29T04:09:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.957+0000 7fdcc8856640 1 --2- 172.21.15.97:0/257007296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc4070990 0x7fdcc409b930 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009870 tx=0x7fdcc002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5beaee4749d8dc0d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.957+0000 7fdcbb7fe640 1 -- 172.21.15.97:0/257007296 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0035020 con 0x7fdcc4070990 2024-03-29T04:09:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.957+0000 7fdcbb7fe640 1 -- 172.21.15.97:0/257007296 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc0031640 con 0x7fdcc4070990 2024-03-29T04:09:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.957+0000 7fdcbb7fe640 1 -- 172.21.15.97:0/257007296 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc002d040 con 0x7fdcc4070990 2024-03-29T04:09:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.957+0000 7fdccaaba640 1 -- 172.21.15.97:0/257007296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc4070990 msgr2=0x7fdcc409b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.957+0000 7fdccaaba640 1 --2- 172.21.15.97:0/257007296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc4070990 0x7fdcc409b930 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009870 tx=0x7fdcc002b8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.957+0000 7fdccaaba640 1 -- 172.21.15.97:0/257007296 shutdown_connections 2024-03-29T04:09:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.957+0000 7fdccaaba640 1 --2- 172.21.15.97:0/257007296 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc40d05c0 0x7fdcc409aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.957+0000 7fdccaaba640 1 --2- 172.21.15.97:0/257007296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc4070990 0x7fdcc409b930 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.957+0000 7fdccaaba640 1 -- 172.21.15.97:0/257007296 >> 172.21.15.97:0/257007296 conn(0x7fdcc40d2430 msgr2=0x7fdcc40d2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:24.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.957+0000 7fdccaaba640 1 -- 172.21.15.97:0/257007296 shutdown_connections 2024-03-29T04:09:24.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.957+0000 7fdccaaba640 1 -- 172.21.15.97:0/257007296 wait complete. 2024-03-29T04:09:24.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.958+0000 7fdccaaba640 1 Processor -- start 2024-03-29T04:09:24.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.959+0000 7fdccaaba640 1 -- start start 2024-03-29T04:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.959+0000 7fdccaaba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcc4070990 0x7fdcc40b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.959+0000 7fdccaaba640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc409af70 0x7fdcc414d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.959+0000 7fdccaaba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc40d05c0 0x7fdcc40c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.959+0000 7fdccaaba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc40d0ba0 con 0x7fdcc40d05c0 2024-03-29T04:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.959+0000 7fdccaaba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc40d0a20 con 0x7fdcc4070990 2024-03-29T04:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.959+0000 7fdccaaba640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc4070d70 con 0x7fdcc409af70 2024-03-29T04:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.959+0000 7fdcc9057640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc40d05c0 0x7fdcc40c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdcc9057640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc40d05c0 0x7fdcc40c8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42322/0 (socket says 172.21.15.97:42322) 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdcc9057640 1 -- 172.21.15.97:0/2409647950 learned_addr learned my addr 172.21.15.97:0/2409647950 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdcc8856640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcc4070990 0x7fdcc40b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdcbbfff640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc409af70 0x7fdcc414d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdcc9057640 1 -- 172.21.15.97:0/2409647950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc409af70 msgr2=0x7fdcc414d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdcc9057640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc409af70 0x7fdcc414d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdcc9057640 1 -- 172.21.15.97:0/2409647950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcc4070990 msgr2=0x7fdcc40b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdcc9057640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcc4070990 0x7fdcc40b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdcc9057640 1 -- 172.21.15.97:0/2409647950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc40b5600 con 0x7fdcc40d05c0 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdcc9057640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc40d05c0 0x7fdcc40c8b20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00c7d0 tx=0x7fdcbc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdcb97fa640 1 -- 172.21.15.97:0/2409647950 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc016020 con 0x7fdcc40d05c0 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdcb97fa640 1 -- 172.21.15.97:0/2409647950 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc00a9f0 con 0x7fdcc40d05c0 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdccaaba640 1 -- 172.21.15.97:0/2409647950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc40b58e0 con 0x7fdcc40d05c0 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdcb97fa640 1 -- 172.21.15.97:0/2409647950 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc00ade0 con 0x7fdcc40d05c0 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.960+0000 7fdccaaba640 1 -- 172.21.15.97:0/2409647950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc40b5ea0 con 0x7fdcc40d05c0 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.961+0000 7fdccaaba640 1 -- 172.21.15.97:0/2409647950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc4068640 con 0x7fdcc40d05c0 2024-03-29T04:09:24.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.962+0000 7fdcb97fa640 1 -- 172.21.15.97:0/2409647950 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdcbc019020 con 0x7fdcc40d05c0 2024-03-29T04:09:24.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.962+0000 7fdcb97fa640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdcb006c530 0x7fdcb006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:24.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.962+0000 7fdcb97fa640 1 -- 172.21.15.97:0/2409647950 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fdcbc08b260 con 0x7fdcc40d05c0 2024-03-29T04:09:24.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.962+0000 7fdcc8856640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdcb006c530 0x7fdcb006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:24.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.963+0000 7fdcc8856640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdcb006c530 0x7fdcb006e9f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0036000 tx=0x7fdcc003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:24.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:24.971+0000 7fdcb97fa640 1 -- 172.21.15.97:0/2409647950 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdcbc020070 con 0x7fdcc40d05c0 2024-03-29T04:09:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.205+0000 7fdccaaba640 1 -- 172.21.15.97:0/2409647950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcc40afda0 con 0x7fdcc40d05c0 2024-03-29T04:09:25.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.206+0000 7fdcb97fa640 1 -- 172.21.15.97:0/2409647950 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16669 (secure 0 0 0) 0x7fdcbc059230 con 0x7fdcc40d05c0 2024-03-29T04:09:25.185 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:25.186 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":32,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:08:31.615419+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:08:28.403360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T04:08:28.909709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.208+0000 7fdccaaba640 1 -- 172.21.15.97:0/2409647950 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdcb006c530 msgr2=0x7fdcb006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.208+0000 7fdccaaba640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdcb006c530 0x7fdcb006e9f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0036000 tx=0x7fdcc003c040 comp rx=0 tx=0).stop 2024-03-29T04:09:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.208+0000 7fdccaaba640 1 -- 172.21.15.97:0/2409647950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc40d05c0 msgr2=0x7fdcc40c8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.208+0000 7fdccaaba640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc40d05c0 0x7fdcc40c8b20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00c7d0 tx=0x7fdcbc00cc90 comp rx=0 tx=0).stop 2024-03-29T04:09:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.208+0000 7fdccaaba640 1 -- 172.21.15.97:0/2409647950 shutdown_connections 2024-03-29T04:09:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.208+0000 7fdccaaba640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdcb006c530 0x7fdcb006e9f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.208+0000 7fdccaaba640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc40d05c0 0x7fdcc40c8b20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.208+0000 7fdccaaba640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdcc409af70 0x7fdcc414d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.208+0000 7fdccaaba640 1 --2- 172.21.15.97:0/2409647950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdcc4070990 0x7fdcc40b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.208+0000 7fdccaaba640 1 -- 172.21.15.97:0/2409647950 >> 172.21.15.97:0/2409647950 conn(0x7fdcc40d2430 msgr2=0x7fdcc40af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.208+0000 7fdccaaba640 1 -- 172.21.15.97:0/2409647950 shutdown_connections 2024-03-29T04:09:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.208+0000 7fdccaaba640 1 -- 172.21.15.97:0/2409647950 wait complete. 2024-03-29T04:09:25.199 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:08:28.403360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:08:28.909709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:09:25.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:25.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.326+0000 7f105a288640 1 Processor -- start 2024-03-29T04:09:25.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.326+0000 7f105a288640 1 -- start start 2024-03-29T04:09:25.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f105a288640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105415ef40 0x7f10540cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f105a288640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10540d03f0 con 0x7f10540ce4a0 2024-03-29T04:09:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f105a288640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10540cebd0 con 0x7f10540ccf80 2024-03-29T04:09:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f105a288640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f10540cc1c0 con 0x7f105415ef40 2024-03-29T04:09:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f1058825640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f10540ce4a0 0x7f10540b41d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51324/0 (socket says 172.21.15.97:51324) 2024-03-29T04:09:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f1058825640 1 -- 172.21.15.97:0/3609822202 learned_addr learned my addr 172.21.15.97:0/3609822202 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f1053fff640 1 --2- 172.21.15.97:0/3609822202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105415ef40 0x7f10540cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f1052ffd640 1 -- 172.21.15.97:0/3609822202 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2103656707 0 0) 0x7f104c001490 con 0x7f10540ce4a0 2024-03-29T04:09:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f1052ffd640 1 -- 172.21.15.97:0/3609822202 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f103c003520 con 0x7f10540ce4a0 2024-03-29T04:09:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f1052ffd640 1 -- 172.21.15.97:0/3609822202 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2484072241 0 0) 0x7f10540cebd0 con 0x7f10540ccf80 2024-03-29T04:09:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f1052ffd640 1 -- 172.21.15.97:0/3609822202 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f104c001490 con 0x7f10540ccf80 2024-03-29T04:09:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f1052ffd640 1 -- 172.21.15.97:0/3609822202 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3862849109 0 0) 0x7f103c003520 con 0x7f10540ce4a0 2024-03-29T04:09:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f1053fff640 1 -- 172.21.15.97:0/3609822202 >> v1:172.21.15.154:6789/0 conn(0x7f10540ccf80 legacy=0x7f10540cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f1053fff640 1 -- 172.21.15.97:0/3609822202 >> v1:172.21.15.97:6789/0 conn(0x7f10540ce4a0 legacy=0x7f10540b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.327+0000 7f1053fff640 1 -- 172.21.15.97:0/3609822202 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10540cc4a0 con 0x7f105415ef40 2024-03-29T04:09:25.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.328+0000 7f1053fff640 1 --2- 172.21.15.97:0/3609822202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105415ef40 0x7f10540cfe10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f10480097d0 tx=0x7f104802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a903d15b4f3ecfdc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:25.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.328+0000 7f1052ffd640 1 -- 172.21.15.97:0/3609822202 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048037070 con 0x7f105415ef40 2024-03-29T04:09:25.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.328+0000 7f1052ffd640 1 -- 172.21.15.97:0/3609822202 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1048030510 con 0x7f105415ef40 2024-03-29T04:09:25.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.328+0000 7f1052ffd640 1 -- 172.21.15.97:0/3609822202 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c001bc0 con 0x7f105415ef40 2024-03-29T04:09:25.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.329+0000 7f105a288640 1 -- 172.21.15.97:0/3609822202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105415ef40 msgr2=0x7f10540cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.329+0000 7f105a288640 1 --2- 172.21.15.97:0/3609822202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105415ef40 0x7f10540cfe10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f10480097d0 tx=0x7f104802b800 comp rx=0 tx=0).stop 2024-03-29T04:09:25.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.329+0000 7f105a288640 1 -- 172.21.15.97:0/3609822202 shutdown_connections 2024-03-29T04:09:25.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.329+0000 7f105a288640 1 --2- 172.21.15.97:0/3609822202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105415ef40 0x7f10540cfe10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.329+0000 7f105a288640 1 -- 172.21.15.97:0/3609822202 >> 172.21.15.97:0/3609822202 conn(0x7f10540b2ed0 msgr2=0x7f10540b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:25.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.329+0000 7f105a288640 1 -- 172.21.15.97:0/3609822202 shutdown_connections 2024-03-29T04:09:25.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.329+0000 7f105a288640 1 -- 172.21.15.97:0/3609822202 wait complete. 2024-03-29T04:09:25.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.330+0000 7f105a288640 1 Processor -- start 2024-03-29T04:09:25.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.330+0000 7f105a288640 1 -- start start 2024-03-29T04:09:25.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f105a288640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10540ccf80 0x7f105414e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f105a288640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10540ce4a0 0x7f10540c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f105a288640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105415ef40 0x7f105415d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f105a288640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f10540b39f0 con 0x7f10540ce4a0 2024-03-29T04:09:25.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f105a288640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f10540cca60 con 0x7f10540ccf80 2024-03-29T04:09:25.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f105a288640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f10540b3b70 con 0x7f105415ef40 2024-03-29T04:09:25.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f10537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10540ce4a0 0x7f10540c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f10537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10540ce4a0 0x7f10540c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42324/0 (socket says 172.21.15.97:42324) 2024-03-29T04:09:25.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f10537fe640 1 -- 172.21.15.97:0/1497517173 learned_addr learned my addr 172.21.15.97:0/1497517173 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:25.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f1053fff640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10540ccf80 0x7f105414e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f1058825640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105415ef40 0x7f105415d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f10537fe640 1 -- 172.21.15.97:0/1497517173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105415ef40 msgr2=0x7f105415d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f10537fe640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105415ef40 0x7f105415d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f10537fe640 1 -- 172.21.15.97:0/1497517173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10540ccf80 msgr2=0x7f105414e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f10537fe640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10540ccf80 0x7f105414e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.331+0000 7f10537fe640 1 -- 172.21.15.97:0/1497517173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10540ca0c0 con 0x7f10540ce4a0 2024-03-29T04:09:25.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.332+0000 7f10537fe640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10540ce4a0 0x7f10540c9700 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f104400c4c0 tx=0x7f104400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:25.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.332+0000 7f1050ff9640 1 -- 172.21.15.97:0/1497517173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10440052d0 con 0x7f10540ce4a0 2024-03-29T04:09:25.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.332+0000 7f1050ff9640 1 -- 172.21.15.97:0/1497517173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1044005470 con 0x7f10540ce4a0 2024-03-29T04:09:25.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.332+0000 7f105a288640 1 -- 172.21.15.97:0/1497517173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10540ca3a0 con 0x7f10540ce4a0 2024-03-29T04:09:25.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.332+0000 7f1050ff9640 1 -- 172.21.15.97:0/1497517173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044012870 con 0x7f10540ce4a0 2024-03-29T04:09:25.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.332+0000 7f105a288640 1 -- 172.21.15.97:0/1497517173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10540ca960 con 0x7f10540ce4a0 2024-03-29T04:09:25.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.333+0000 7f105a288640 1 -- 172.21.15.97:0/1497517173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f105409f530 con 0x7f10540ce4a0 2024-03-29T04:09:25.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.334+0000 7f1050ff9640 1 -- 172.21.15.97:0/1497517173 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1044012a10 con 0x7f10540ce4a0 2024-03-29T04:09:25.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.334+0000 7f1050ff9640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f104006c530 0x7f104006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.334+0000 7f1053fff640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f104006c530 0x7f104006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.334+0000 7f1050ff9640 1 -- 172.21.15.97:0/1497517173 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f104408b1f0 con 0x7f10540ce4a0 2024-03-29T04:09:25.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.335+0000 7f1053fff640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f104006c530 0x7f104006e9f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1048034000 tx=0x7f1048008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:25.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.339+0000 7f1050ff9640 1 -- 172.21.15.97:0/1497517173 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f10440591c0 con 0x7f10540ce4a0 2024-03-29T04:09:25.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.518+0000 7f105a288640 1 -- 172.21.15.97:0/1497517173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10540cdb70 con 0x7f10540ce4a0 2024-03-29T04:09:25.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.519+0000 7f1050ff9640 1 -- 172.21.15.97:0/1497517173 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) ==== 93+0+10 (secure 0 0 0) 0x7f104405d4c0 con 0x7f10540ce4a0 2024-03-29T04:09:25.499 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:09:25.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.521+0000 7f105a288640 1 -- 172.21.15.97:0/1497517173 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f104006c530 msgr2=0x7f104006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.521+0000 7f105a288640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f104006c530 0x7f104006e9f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1048034000 tx=0x7f1048008030 comp rx=0 tx=0).stop 2024-03-29T04:09:25.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.521+0000 7f105a288640 1 -- 172.21.15.97:0/1497517173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10540ce4a0 msgr2=0x7f10540c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.521+0000 7f105a288640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10540ce4a0 0x7f10540c9700 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f104400c4c0 tx=0x7f104400c980 comp rx=0 tx=0).stop 2024-03-29T04:09:25.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.521+0000 7f105a288640 1 -- 172.21.15.97:0/1497517173 shutdown_connections 2024-03-29T04:09:25.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.521+0000 7f105a288640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f104006c530 0x7f104006e9f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.521+0000 7f105a288640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f105415ef40 0x7f105415d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.521+0000 7f105a288640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10540ce4a0 0x7f10540c9700 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.521+0000 7f105a288640 1 --2- 172.21.15.97:0/1497517173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10540ccf80 0x7f105414e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.521+0000 7f105a288640 1 -- 172.21.15.97:0/1497517173 >> 172.21.15.97:0/1497517173 conn(0x7f10540b2ed0 msgr2=0x7f105406ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:25.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.521+0000 7f105a288640 1 -- 172.21.15.97:0/1497517173 shutdown_connections 2024-03-29T04:09:25.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.521+0000 7f105a288640 1 -- 172.21.15.97:0/1497517173 wait complete. 2024-03-29T04:09:25.512 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:09:25.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.640+0000 7fb8b941b640 1 Processor -- start 2024-03-29T04:09:25.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.640+0000 7fb8b941b640 1 -- start start 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.640+0000 7fb8b941b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8b415e920 0x7fb8b40d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.640+0000 7fb8b941b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8b40d3780 con 0x7fb8b40d16a0 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.640+0000 7fb8b941b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8b415ed00 con 0x7fb8b40cfde0 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.640+0000 7fb8b941b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fb8b40d21e0 con 0x7fb8b415e920 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.641+0000 7fb8b2ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8b415e920 0x7fb8b40d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.641+0000 7fb8b37fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb8b40d16a0 0x7fb8b406c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51338/0 (socket says 172.21.15.97:51338) 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.641+0000 7fb8b2ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8b415e920 0x7fb8b40d3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42326/0 (socket says 172.21.15.97:42326) 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.641+0000 7fb8b37fe640 1 -- 172.21.15.97:0/1751076878 learned_addr learned my addr 172.21.15.97:0/1751076878 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.641+0000 7fb8b2ffd640 1 -- 172.21.15.97:0/1751076878 >> v1:172.21.15.154:6789/0 conn(0x7fb8b40cfde0 legacy=0x7fb8b40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.641+0000 7fb8b2ffd640 1 -- 172.21.15.97:0/1751076878 >> v1:172.21.15.97:6789/0 conn(0x7fb8b40d16a0 legacy=0x7fb8b406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.641+0000 7fb8b2ffd640 1 -- 172.21.15.97:0/1751076878 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b40d2480 con 0x7fb8b415e920 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.641+0000 7fb8b27fc640 1 -- 172.21.15.97:0/1751076878 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3338285529 0 0) 0x7fb8ac001490 con 0x7fb8b40d16a0 2024-03-29T04:09:25.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.642+0000 7fb8b2ffd640 1 --2- 172.21.15.97:0/1751076878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b415e920 0x7fb8b40d3240 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40097a0 tx=0x7fb8a402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bc13be9a7b9e58 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:25.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.642+0000 7fb8b27fc640 1 -- 172.21.15.97:0/1751076878 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4035020 con 0x7fb8b415e920 2024-03-29T04:09:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.642+0000 7fb8b27fc640 1 -- 172.21.15.97:0/1751076878 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8a4031830 con 0x7fb8b415e920 2024-03-29T04:09:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.642+0000 7fb8b27fc640 1 -- 172.21.15.97:0/1751076878 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4031c00 con 0x7fb8b415e920 2024-03-29T04:09:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.643+0000 7fb8b941b640 1 -- 172.21.15.97:0/1751076878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b415e920 msgr2=0x7fb8b40d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.643+0000 7fb8b941b640 1 --2- 172.21.15.97:0/1751076878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b415e920 0x7fb8b40d3240 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40097a0 tx=0x7fb8a402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.643+0000 7fb8b941b640 1 -- 172.21.15.97:0/1751076878 shutdown_connections 2024-03-29T04:09:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.643+0000 7fb8b941b640 1 --2- 172.21.15.97:0/1751076878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b415e920 0x7fb8b40d3240 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.643+0000 7fb8b941b640 1 -- 172.21.15.97:0/1751076878 >> 172.21.15.97:0/1751076878 conn(0x7fb8b408bfc0 msgr2=0x7fb8b40d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.643+0000 7fb8b941b640 1 -- 172.21.15.97:0/1751076878 shutdown_connections 2024-03-29T04:09:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.643+0000 7fb8b941b640 1 -- 172.21.15.97:0/1751076878 wait complete. 2024-03-29T04:09:25.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.644+0000 7fb8b941b640 1 Processor -- start 2024-03-29T04:09:25.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.644+0000 7fb8b941b640 1 -- start start 2024-03-29T04:09:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8b941b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8b40cfde0 0x7fb8b40b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8b941b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b40d16a0 0x7fb8b40b92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8b941b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8b415e920 0x7fb8b4155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8b941b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b40d2d20 con 0x7fb8b40d16a0 2024-03-29T04:09:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8b941b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b40d2a40 con 0x7fb8b415e920 2024-03-29T04:09:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8b941b640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b40d4ef0 con 0x7fb8b40cfde0 2024-03-29T04:09:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8abfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b40d16a0 0x7fb8b40b92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8abfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b40d16a0 0x7fb8b40b92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42332/0 (socket says 172.21.15.97:42332) 2024-03-29T04:09:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8abfff640 1 -- 172.21.15.97:0/71746726 learned_addr learned my addr 172.21.15.97:0/71746726 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8b2ffd640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8b40cfde0 0x7fb8b40b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8b37fe640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8b415e920 0x7fb8b4155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8abfff640 1 -- 172.21.15.97:0/71746726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8b40cfde0 msgr2=0x7fb8b40b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8abfff640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8b40cfde0 0x7fb8b40b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8abfff640 1 -- 172.21.15.97:0/71746726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8b415e920 msgr2=0x7fb8b4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8abfff640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8b415e920 0x7fb8b4155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8abfff640 1 -- 172.21.15.97:0/71746726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b41f9b10 con 0x7fb8b40d16a0 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8b2ffd640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8b40cfde0 0x7fb8b40b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8b37fe640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8b415e920 0x7fb8b4155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8abfff640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b40d16a0 0x7fb8b40b92d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00b3f0 tx=0x7fb89c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8ab7fe640 1 -- 172.21.15.97:0/71746726 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c015020 con 0x7fb8b40d16a0 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8ab7fe640 1 -- 172.21.15.97:0/71746726 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb89c011830 con 0x7fb8b40d16a0 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8b941b640 1 -- 172.21.15.97:0/71746726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b40b9f80 con 0x7fb8b40d16a0 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8ab7fe640 1 -- 172.21.15.97:0/71746726 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c011c20 con 0x7fb8b40d16a0 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.645+0000 7fb8b941b640 1 -- 172.21.15.97:0/71746726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b41f9030 con 0x7fb8b40d16a0 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.646+0000 7fb8b941b640 1 -- 172.21.15.97:0/71746726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b40a2560 con 0x7fb8b40d16a0 2024-03-29T04:09:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.647+0000 7fb8ab7fe640 1 -- 172.21.15.97:0/71746726 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb89c018020 con 0x7fb8b40d16a0 2024-03-29T04:09:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.647+0000 7fb8ab7fe640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb89406c530 0x7fb89406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.647+0000 7fb8ab7fe640 1 -- 172.21.15.97:0/71746726 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb89c08b2e0 con 0x7fb8b40d16a0 2024-03-29T04:09:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.647+0000 7fb8b2ffd640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb89406c530 0x7fb89406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.648+0000 7fb8b2ffd640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb89406c530 0x7fb89406e9f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4036000 tx=0x7fb8a4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:25.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.652+0000 7fb8ab7fe640 1 -- 172.21.15.97:0/71746726 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb89c01f030 con 0x7fb8b40d16a0 2024-03-29T04:09:25.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.869+0000 7fb8b941b640 1 -- 172.21.15.97:0/71746726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb8b404f9d0 con 0x7fb8b40d16a0 2024-03-29T04:09:25.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.869+0000 7fb8ab7fe640 1 -- 172.21.15.97:0/71746726 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) ==== 104+0+11 (secure 0 0 0) 0x7fb89c0172c0 con 0x7fb8b40d16a0 2024-03-29T04:09:25.849 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:09:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.872+0000 7fb8b941b640 1 -- 172.21.15.97:0/71746726 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb89406c530 msgr2=0x7fb89406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.872+0000 7fb8b941b640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb89406c530 0x7fb89406e9f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4036000 tx=0x7fb8a4008030 comp rx=0 tx=0).stop 2024-03-29T04:09:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.872+0000 7fb8b941b640 1 -- 172.21.15.97:0/71746726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b40d16a0 msgr2=0x7fb8b40b92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.872+0000 7fb8b941b640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b40d16a0 0x7fb8b40b92d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00b3f0 tx=0x7fb89c00b8b0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.872+0000 7fb8b941b640 1 -- 172.21.15.97:0/71746726 shutdown_connections 2024-03-29T04:09:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.872+0000 7fb8b941b640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb89406c530 0x7fb89406e9f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.872+0000 7fb8b941b640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb8b415e920 0x7fb8b4155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.872+0000 7fb8b941b640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b40d16a0 0x7fb8b40b92d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.872+0000 7fb8b941b640 1 --2- 172.21.15.97:0/71746726 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb8b40cfde0 0x7fb8b40b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.872+0000 7fb8b941b640 1 -- 172.21.15.97:0/71746726 >> 172.21.15.97:0/71746726 conn(0x7fb8b408bfc0 msgr2=0x7fb8b4156c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.872+0000 7fb8b941b640 1 -- 172.21.15.97:0/71746726 shutdown_connections 2024-03-29T04:09:25.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.872+0000 7fb8b941b640 1 -- 172.21.15.97:0/71746726 wait complete. 2024-03-29T04:09:25.862 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:09:25.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.989+0000 7f2eb6f11640 1 Processor -- start 2024-03-29T04:09:25.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.989+0000 7f2eb6f11640 1 -- start start 2024-03-29T04:09:25.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eb6f11640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2eb00b4e40 0x7f2eb00c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eb6f11640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2eb00b31c0 0x7f2eb00b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eb6f11640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eb01604d0 con 0x7f2eb00b2640 2024-03-29T04:09:25.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eb6f11640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2eb00c6d60 con 0x7f2eb00b31c0 2024-03-29T04:09:25.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eb6f11640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb00b2a90 con 0x7f2eb00b4e40 2024-03-29T04:09:25.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eaffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2eb00b31c0 0x7f2eb00b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eb54ae640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2eb00b2640 0x7f2eb00cbec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51348/0 (socket says 172.21.15.97:51348) 2024-03-29T04:09:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eaffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2eb00b31c0 0x7f2eb00b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42342/0 (socket says 172.21.15.97:42342) 2024-03-29T04:09:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eb54ae640 1 -- 172.21.15.97:0/1242417962 learned_addr learned my addr 172.21.15.97:0/1242417962 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eb4cad640 1 --2- 172.21.15.97:0/1242417962 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2eb00b4e40 0x7f2eb00c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eaffff640 1 -- 172.21.15.97:0/1242417962 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2eb00b4e40 msgr2=0x7f2eb00c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eaffff640 1 --2- 172.21.15.97:0/1242417962 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2eb00b4e40 0x7f2eb00c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eaffff640 1 -- 172.21.15.97:0/1242417962 >> v1:172.21.15.97:6789/0 conn(0x7f2eb00b2640 legacy=0x7f2eb00cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eaffff640 1 -- 172.21.15.97:0/1242417962 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb01607b0 con 0x7f2eb00b31c0 2024-03-29T04:09:25.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.990+0000 7f2eaf7fe640 1 -- 172.21.15.97:0/1242417962 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 116642705 0 0) 0x7f2eb01604d0 con 0x7f2eb00b2640 2024-03-29T04:09:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.992+0000 7f2eaffff640 1 --2- 172.21.15.97:0/1242417962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb00b31c0 0x7f2eb00b2100 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00097a0 tx=0x7f2ea002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b3f126086bd26c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.992+0000 7f2eaf7fe640 1 -- 172.21.15.97:0/1242417962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea00315a0 con 0x7f2eb00b31c0 2024-03-29T04:09:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.992+0000 7f2eaf7fe640 1 -- 172.21.15.97:0/1242417962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea0037070 con 0x7f2eb00b31c0 2024-03-29T04:09:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.992+0000 7f2eaf7fe640 1 -- 172.21.15.97:0/1242417962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0031960 con 0x7f2eb00b31c0 2024-03-29T04:09:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.992+0000 7f2eb6f11640 1 -- 172.21.15.97:0/1242417962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb00b31c0 msgr2=0x7f2eb00b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.992+0000 7f2eb6f11640 1 --2- 172.21.15.97:0/1242417962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb00b31c0 0x7f2eb00b2100 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00097a0 tx=0x7f2ea002b840 comp rx=0 tx=0).stop 2024-03-29T04:09:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.992+0000 7f2eb6f11640 1 -- 172.21.15.97:0/1242417962 shutdown_connections 2024-03-29T04:09:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.992+0000 7f2eb6f11640 1 --2- 172.21.15.97:0/1242417962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb00b31c0 0x7f2eb00b2100 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.992+0000 7f2eb6f11640 1 --2- 172.21.15.97:0/1242417962 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2eb00b4e40 0x7f2eb00c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.992+0000 7f2eb6f11640 1 -- 172.21.15.97:0/1242417962 >> 172.21.15.97:0/1242417962 conn(0x7f2eb00d0450 msgr2=0x7f2eb00d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.992+0000 7f2eb6f11640 1 -- 172.21.15.97:0/1242417962 shutdown_connections 2024-03-29T04:09:25.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.992+0000 7f2eb6f11640 1 -- 172.21.15.97:0/1242417962 wait complete. 2024-03-29T04:09:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.993+0000 7f2eb6f11640 1 Processor -- start 2024-03-29T04:09:25.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.993+0000 7f2eb6f11640 1 -- start start 2024-03-29T04:09:25.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.994+0000 7f2eb6f11640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2eb00b2640 0x7f2eb0147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.994+0000 7f2eb6f11640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb00b31c0 0x7f2eb0159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.994+0000 7f2eb6f11640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2eb00b4e40 0x7f2eb01597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.994+0000 7f2eb6f11640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb0160d70 con 0x7f2eb00b31c0 2024-03-29T04:09:25.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.994+0000 7f2eb6f11640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb00cba20 con 0x7f2eb00b4e40 2024-03-29T04:09:25.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.994+0000 7f2eb6f11640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb00b3620 con 0x7f2eb00b2640 2024-03-29T04:09:25.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.994+0000 7f2eaffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb00b31c0 0x7f2eb0159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.994+0000 7f2eaffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb00b31c0 0x7f2eb0159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42352/0 (socket says 172.21.15.97:42352) 2024-03-29T04:09:25.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.994+0000 7f2eaffff640 1 -- 172.21.15.97:0/3602315025 learned_addr learned my addr 172.21.15.97:0/3602315025 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.995+0000 7f2eb4cad640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2eb00b2640 0x7f2eb0147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.995+0000 7f2eaffff640 1 -- 172.21.15.97:0/3602315025 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2eb00b2640 msgr2=0x7f2eb0147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.995+0000 7f2eaffff640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2eb00b2640 0x7f2eb0147e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.995+0000 7f2eaffff640 1 -- 172.21.15.97:0/3602315025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2eb00b4e40 msgr2=0x7f2eb01597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.995+0000 7f2eb54ae640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2eb00b4e40 0x7f2eb01597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.995+0000 7f2eaffff640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2eb00b4e40 0x7f2eb01597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.995+0000 7f2eaffff640 1 -- 172.21.15.97:0/3602315025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb014b700 con 0x7f2eb00b31c0 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.995+0000 7f2eaffff640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb00b31c0 0x7f2eb0159290 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0034040 tx=0x7f2ea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.995+0000 7f2ead7fa640 1 -- 172.21.15.97:0/3602315025 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0032630 con 0x7f2eb00b31c0 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.995+0000 7f2ead7fa640 1 -- 172.21.15.97:0/3602315025 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea00327d0 con 0x7f2eb00b31c0 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.995+0000 7f2eb6f11640 1 -- 172.21.15.97:0/3602315025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb01f38b0 con 0x7f2eb00b31c0 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.995+0000 7f2ead7fa640 1 -- 172.21.15.97:0/3602315025 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0032ba0 con 0x7f2eb00b31c0 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.995+0000 7f2eb6f11640 1 -- 172.21.15.97:0/3602315025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb01f2dd0 con 0x7f2eb00b31c0 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.996+0000 7f2eb6f11640 1 -- 172.21.15.97:0/3602315025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb004f9d0 con 0x7f2eb00b31c0 2024-03-29T04:09:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.997+0000 7f2ead7fa640 1 -- 172.21.15.97:0/3602315025 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2ea004e430 con 0x7f2eb00b31c0 2024-03-29T04:09:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.997+0000 7f2ead7fa640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2e9006c530 0x7f2e9006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.997+0000 7f2eb4cad640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2e9006c530 0x7f2e9006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.997+0000 7f2ead7fa640 1 -- 172.21.15.97:0/3602315025 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2ea002f030 con 0x7f2eb00b31c0 2024-03-29T04:09:25.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:25.997+0000 7f2eb4cad640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2e9006c530 0x7f2e9006e9f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2e98006fd0 tx=0x7f2e98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:25.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.005+0000 7f2ead7fa640 1 -- 172.21.15.97:0/3602315025 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2ea0040070 con 0x7f2eb00b31c0 2024-03-29T04:09:26.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.185+0000 7f2eb6f11640 1 -- 172.21.15.97:0/3602315025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2eb015d690 con 0x7f2eb00b31c0 2024-03-29T04:09:26.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.186+0000 7f2ead7fa640 1 -- 172.21.15.97:0/3602315025 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) ==== 100+0+11 (secure 0 0 0) 0x7f2ea007bc50 con 0x7f2eb00b31c0 2024-03-29T04:09:26.165 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:09:26.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.187+0000 7f2eb6f11640 1 -- 172.21.15.97:0/3602315025 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2e9006c530 msgr2=0x7f2e9006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.187+0000 7f2eb6f11640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2e9006c530 0x7f2e9006e9f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2e98006fd0 tx=0x7f2e98008040 comp rx=0 tx=0).stop 2024-03-29T04:09:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.187+0000 7f2eb6f11640 1 -- 172.21.15.97:0/3602315025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb00b31c0 msgr2=0x7f2eb0159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.187+0000 7f2eb6f11640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb00b31c0 0x7f2eb0159290 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0034040 tx=0x7f2ea00046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.188+0000 7f2eb6f11640 1 -- 172.21.15.97:0/3602315025 shutdown_connections 2024-03-29T04:09:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.188+0000 7f2eb6f11640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2e9006c530 0x7f2e9006e9f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.188+0000 7f2eb6f11640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2eb00b4e40 0x7f2eb01597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.188+0000 7f2eb6f11640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb00b31c0 0x7f2eb0159290 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.188+0000 7f2eb6f11640 1 --2- 172.21.15.97:0/3602315025 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2eb00b2640 0x7f2eb0147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.188+0000 7f2eb6f11640 1 -- 172.21.15.97:0/3602315025 >> 172.21.15.97:0/3602315025 conn(0x7f2eb00d0450 msgr2=0x7f2eb015e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.188+0000 7f2eb6f11640 1 -- 172.21.15.97:0/3602315025 shutdown_connections 2024-03-29T04:09:26.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.188+0000 7f2eb6f11640 1 -- 172.21.15.97:0/3602315025 wait complete. 2024-03-29T04:09:26.178 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:09:26.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.300+0000 7fe583622640 1 Processor -- start 2024-03-29T04:09:26.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.300+0000 7fe583622640 1 -- start start 2024-03-29T04:09:26.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.301+0000 7fe583622640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe57c15e780 0x7fe57c0cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.301+0000 7fe583622640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe57c15eb60 0x7fe57c0cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.301+0000 7fe583622640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe57c0cfe70 con 0x7fe57c0cf640 2024-03-29T04:09:26.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.301+0000 7fe583622640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fe57c0b7f50 con 0x7fe57c15eb60 2024-03-29T04:09:26.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.301+0000 7fe583622640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe57c0b8230 con 0x7fe57c15e780 2024-03-29T04:09:26.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.301+0000 7fe580bbd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe57c15eb60 0x7fe57c0cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.301+0000 7fe580bbd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe57c15eb60 0x7fe57c0cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42358/0 (socket says 172.21.15.97:42358) 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.301+0000 7fe580bbd640 1 -- 172.21.15.97:0/2489407546 learned_addr learned my addr 172.21.15.97:0/2489407546 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.301+0000 7fe5813be640 1 --2- 172.21.15.97:0/2489407546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe57c15e780 0x7fe57c0cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.301+0000 7fe580bbd640 1 -- 172.21.15.97:0/2489407546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe57c15e780 msgr2=0x7fe57c0cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.301+0000 7fe580bbd640 1 --2- 172.21.15.97:0/2489407546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe57c15e780 0x7fe57c0cebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.301+0000 7fe580bbd640 1 -- 172.21.15.97:0/2489407546 >> v1:172.21.15.154:6789/0 conn(0x7fe57c0cf640 legacy=0x7fe57c0b8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.302+0000 7fe580bbd640 1 -- 172.21.15.97:0/2489407546 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57c0b8510 con 0x7fe57c15eb60 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.302+0000 7fe580bbd640 1 --2- 172.21.15.97:0/2489407546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15eb60 0x7fe57c0cf100 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009870 tx=0x7fe56c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a31619297377b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.302+0000 7fe573fff640 1 -- 172.21.15.97:0/2489407546 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c034070 con 0x7fe57c15eb60 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.302+0000 7fe573fff640 1 -- 172.21.15.97:0/2489407546 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56c030840 con 0x7fe57c15eb60 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.302+0000 7fe573fff640 1 -- 172.21.15.97:0/2489407546 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c030c10 con 0x7fe57c15eb60 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.303+0000 7fe583622640 1 -- 172.21.15.97:0/2489407546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15eb60 msgr2=0x7fe57c0cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.303+0000 7fe583622640 1 --2- 172.21.15.97:0/2489407546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15eb60 0x7fe57c0cf100 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009870 tx=0x7fe56c02b980 comp rx=0 tx=0).stop 2024-03-29T04:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.303+0000 7fe583622640 1 -- 172.21.15.97:0/2489407546 shutdown_connections 2024-03-29T04:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.303+0000 7fe583622640 1 --2- 172.21.15.97:0/2489407546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15eb60 0x7fe57c0cf100 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.303+0000 7fe583622640 1 --2- 172.21.15.97:0/2489407546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe57c15e780 0x7fe57c0cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.303+0000 7fe583622640 1 -- 172.21.15.97:0/2489407546 >> 172.21.15.97:0/2489407546 conn(0x7fe57c08bfc0 msgr2=0x7fe57c0d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.303+0000 7fe583622640 1 -- 172.21.15.97:0/2489407546 shutdown_connections 2024-03-29T04:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.303+0000 7fe583622640 1 -- 172.21.15.97:0/2489407546 wait complete. 2024-03-29T04:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.305+0000 7fe583622640 1 Processor -- start 2024-03-29T04:09:26.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.305+0000 7fe583622640 1 -- start start 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe583622640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe57c0cf640 0x7fe57c157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe583622640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15e780 0x7fe57c1f7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe583622640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe57c0cdda0 0x7fe57c1f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe583622640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c0d3580 con 0x7fe57c15e780 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe583622640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe57c0b8ad0 con 0x7fe57c0cdda0 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe583622640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fe57c0d3700 con 0x7fe57c0cf640 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe580bbd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15e780 0x7fe57c1f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe580bbd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15e780 0x7fe57c1f7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42370/0 (socket says 172.21.15.97:42370) 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe580bbd640 1 -- 172.21.15.97:0/1724028076 learned_addr learned my addr 172.21.15.97:0/1724028076 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe5813be640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe57c0cf640 0x7fe57c157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe581bbf640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe57c0cdda0 0x7fe57c1f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe580bbd640 1 -- 172.21.15.97:0/1724028076 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe57c0cf640 msgr2=0x7fe57c157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe580bbd640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe57c0cf640 0x7fe57c157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe580bbd640 1 -- 172.21.15.97:0/1724028076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe57c0cdda0 msgr2=0x7fe57c1f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe580bbd640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe57c0cdda0 0x7fe57c1f42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.306+0000 7fe580bbd640 1 -- 172.21.15.97:0/1724028076 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57c1f8060 con 0x7fe57c15e780 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.307+0000 7fe580bbd640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15e780 0x7fe57c1f7810 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe56c000c00 tx=0x7fe56c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.307+0000 7fe571ffb640 1 -- 172.21.15.97:0/1724028076 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c034040 con 0x7fe57c15e780 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.307+0000 7fe571ffb640 1 -- 172.21.15.97:0/1724028076 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56c039d90 con 0x7fe57c15e780 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.307+0000 7fe583622640 1 -- 172.21.15.97:0/1724028076 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57c1f8340 con 0x7fe57c15e780 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.307+0000 7fe571ffb640 1 -- 172.21.15.97:0/1724028076 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c043610 con 0x7fe57c15e780 2024-03-29T04:09:26.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.307+0000 7fe583622640 1 -- 172.21.15.97:0/1724028076 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57c158600 con 0x7fe57c15e780 2024-03-29T04:09:26.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.308+0000 7fe583622640 1 -- 172.21.15.97:0/1724028076 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe57c068640 con 0x7fe57c15e780 2024-03-29T04:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.309+0000 7fe571ffb640 1 -- 172.21.15.97:0/1724028076 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe56c04a060 con 0x7fe57c15e780 2024-03-29T04:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.309+0000 7fe571ffb640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe56006c530 0x7fe56006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.309+0000 7fe5813be640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe56006c530 0x7fe56006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.309+0000 7fe571ffb640 1 -- 172.21.15.97:0/1724028076 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe56c0ad570 con 0x7fe57c15e780 2024-03-29T04:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.310+0000 7fe5813be640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe56006c530 0x7fe56006e9f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe578006fd0 tx=0x7fe578008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.314+0000 7fe571ffb640 1 -- 172.21.15.97:0/1724028076 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe56c03e070 con 0x7fe57c15e780 2024-03-29T04:09:26.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.506+0000 7fe583622640 1 -- 172.21.15.97:0/1724028076 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe57c1587d0 con 0x7fe57c15e780 2024-03-29T04:09:26.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.506+0000 7fe571ffb640 1 -- 172.21.15.97:0/1724028076 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7fe56c07b540 con 0x7fe57c15e780 2024-03-29T04:09:26.486 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-03-29T04:09:26.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.508+0000 7fe583622640 1 -- 172.21.15.97:0/1724028076 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe56006c530 msgr2=0x7fe56006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.508+0000 7fe583622640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe56006c530 0x7fe56006e9f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe578006fd0 tx=0x7fe578008040 comp rx=0 tx=0).stop 2024-03-29T04:09:26.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.508+0000 7fe583622640 1 -- 172.21.15.97:0/1724028076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15e780 msgr2=0x7fe57c1f7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.508+0000 7fe583622640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15e780 0x7fe57c1f7810 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe56c000c00 tx=0x7fe56c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.508+0000 7fe583622640 1 -- 172.21.15.97:0/1724028076 shutdown_connections 2024-03-29T04:09:26.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.508+0000 7fe583622640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fe56006c530 0x7fe56006e9f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.508+0000 7fe583622640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe57c0cdda0 0x7fe57c1f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.508+0000 7fe583622640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe57c15e780 0x7fe57c1f7810 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.508+0000 7fe583622640 1 --2- 172.21.15.97:0/1724028076 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe57c0cf640 0x7fe57c157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.508+0000 7fe583622640 1 -- 172.21.15.97:0/1724028076 >> 172.21.15.97:0/1724028076 conn(0x7fe57c08bfc0 msgr2=0x7fe57c0b9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:26.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.508+0000 7fe583622640 1 -- 172.21.15.97:0/1724028076 shutdown_connections 2024-03-29T04:09:26.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.508+0000 7fe583622640 1 -- 172.21.15.97:0/1724028076 wait complete. 2024-03-29T04:09:26.502 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-29T04:09:26.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:26.503 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-29T04:09:26.562 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T04:09:26.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:26.562 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T04:09:26.588 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:09:26.589 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:09:26.589 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:26.607 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:09:26.607 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:26.664 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:26.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.787+0000 7f859f7a8640 1 Processor -- start 2024-03-29T04:09:26.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.788+0000 7f859f7a8640 1 -- start start 2024-03-29T04:09:26.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.788+0000 7f859f7a8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f859815b370 0x7f85980b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.788+0000 7f859f7a8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f859806f3f0 0x7f8598072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.788+0000 7f859f7a8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8598071ae0 con 0x7f85980730c0 2024-03-29T04:09:26.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.788+0000 7f859f7a8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f8598071dc0 con 0x7f859815b370 2024-03-29T04:09:26.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.788+0000 7f859f7a8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f85980720a0 con 0x7f859806f3f0 2024-03-29T04:09:26.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.788+0000 7f859dd45640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f85980730c0 0x7f85980ae020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51362/0 (socket says 172.21.15.97:51362) 2024-03-29T04:09:26.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.788+0000 7f859dd45640 1 -- 172.21.15.97:0/1253366136 learned_addr learned my addr 172.21.15.97:0/1253366136 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:26.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f859cd43640 1 --2- 172.21.15.97:0/1253366136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f859806f3f0 0x7f8598072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3849832136 0 0) 0x7f8598071ae0 con 0x7f85980730c0 2024-03-29T04:09:26.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8584005730 con 0x7f85980730c0 2024-03-29T04:09:26.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f859d544640 1 --2- 172.21.15.97:0/1253366136 >> v2:172.21.15.154:3300/0 conn(0x7f859815b370 0x7f85980b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 131892490 0 0) 0x7f8584005730 con 0x7f85980730c0 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f859806f3f0 msgr2=0x7f8598072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 --2- 172.21.15.97:0/1253366136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f859806f3f0 0x7f8598072b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 >> v2:172.21.15.154:3300/0 conn(0x7f859815b370 msgr2=0x7f85980b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 --2- 172.21.15.97:0/1253366136 >> v2:172.21.15.154:3300/0 conn(0x7f859815b370 0x7f85980b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8598072380 con 0x7f85980730c0 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f85900019f0 con 0x7f85980730c0 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 >> v1:172.21.15.97:6789/0 conn(0x7f85980730c0 legacy=0x7f85980ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 --2- 172.21.15.97:0/1253366136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8584005bc0 0x7f8584005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 --2- 172.21.15.97:0/1253366136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8584006a10 0x7f8584006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 --2- 172.21.15.97:0/1253366136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f859806f3f0 0x7f85840073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8598071dc0 con 0x7f8584005bc0 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f85980720a0 con 0x7f8584006a10 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f858400dbd0 con 0x7f859806f3f0 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f859d544640 1 --2- 172.21.15.97:0/1253366136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8584005bc0 0x7f8584005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f859cd43640 1 --2- 172.21.15.97:0/1253366136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8584006a10 0x7f8584006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f859dd45640 1 --2- 172.21.15.97:0/1253366136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f859806f3f0 0x7f85840073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f859d544640 1 -- 172.21.15.97:0/1253366136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f859806f3f0 msgr2=0x7f85840073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:26.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f859d544640 1 --2- 172.21.15.97:0/1253366136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f859806f3f0 0x7f85840073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f859d544640 1 -- 172.21.15.97:0/1253366136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8584006a10 msgr2=0x7f8584006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:26.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.789+0000 7f859d544640 1 --2- 172.21.15.97:0/1253366136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8584006a10 0x7f8584006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.790+0000 7f859d544640 1 -- 172.21.15.97:0/1253366136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8584005730 con 0x7f8584005bc0 2024-03-29T04:09:26.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.791+0000 7f859d544640 1 --2- 172.21.15.97:0/1253366136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8584005bc0 0x7f8584005fe0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8588009ac0 tx=0x7f8584007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95518dd365289e58 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.791+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588039070 con 0x7f8584005bc0 2024-03-29T04:09:26.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.791+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f858802c710 con 0x7f8584005bc0 2024-03-29T04:09:26.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.791+0000 7f858ffff640 1 -- 172.21.15.97:0/1253366136 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588009690 con 0x7f8584005bc0 2024-03-29T04:09:26.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.791+0000 7f859f7a8640 1 -- 172.21.15.97:0/1253366136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8584005bc0 msgr2=0x7f8584005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.791+0000 7f859f7a8640 1 --2- 172.21.15.97:0/1253366136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8584005bc0 0x7f8584005fe0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8588009ac0 tx=0x7f8584007a40 comp rx=0 tx=0).stop 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.791+0000 7f859dd45640 1 -- 172.21.15.97:0/1253366136 reap_dead start 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.791+0000 7f859f7a8640 1 -- 172.21.15.97:0/1253366136 shutdown_connections 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.791+0000 7f859f7a8640 1 -- 172.21.15.97:0/1253366136 >> 172.21.15.97:0/1253366136 conn(0x7f859806b510 msgr2=0x7f859806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.792+0000 7f859f7a8640 1 -- 172.21.15.97:0/1253366136 shutdown_connections 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.792+0000 7f859f7a8640 1 -- 172.21.15.97:0/1253366136 wait complete. 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.793+0000 7f859f7a8640 1 Processor -- start 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.793+0000 7f859f7a8640 1 -- start start 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859f7a8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8584005bc0 0x7f8598153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859f7a8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85981f0a30 0x7f859815de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859f7a8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8598158e60 0x7f8598155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859f7a8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f85980b3800 con 0x7f8584005bc0 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859f7a8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8598073550 con 0x7f85981f0a30 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859f7a8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f85980b3980 con 0x7f8598158e60 2024-03-29T04:09:26.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859d544640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8584005bc0 0x7f8598153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859d544640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8584005bc0 0x7f8598153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42384/0 (socket says 172.21.15.97:42384) 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859d544640 1 -- 172.21.15.97:0/1618790075 learned_addr learned my addr 172.21.15.97:0/1618790075 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859cd43640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85981f0a30 0x7f859815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859dd45640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8598158e60 0x7f8598155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859d544640 1 -- 172.21.15.97:0/1618790075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8598158e60 msgr2=0x7f8598155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859d544640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8598158e60 0x7f8598155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859d544640 1 -- 172.21.15.97:0/1618790075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85981f0a30 msgr2=0x7f859815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859d544640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85981f0a30 0x7f859815de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859d544640 1 -- 172.21.15.97:0/1618790075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85981f4040 con 0x7f8584005bc0 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859dd45640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8598158e60 0x7f8598155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f859d544640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8584005bc0 0x7f8598153db0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8588037040 tx=0x7f8588004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.794+0000 7f858dffb640 1 -- 172.21.15.97:0/1618790075 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588042020 con 0x7f8584005bc0 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.795+0000 7f858dffb640 1 -- 172.21.15.97:0/1618790075 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f858803a040 con 0x7f8584005bc0 2024-03-29T04:09:26.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.795+0000 7f859f7a8640 1 -- 172.21.15.97:0/1618790075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85981f4320 con 0x7f8584005bc0 2024-03-29T04:09:26.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.795+0000 7f858dffb640 1 -- 172.21.15.97:0/1618790075 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588039070 con 0x7f8584005bc0 2024-03-29T04:09:26.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.795+0000 7f859f7a8640 1 -- 172.21.15.97:0/1618790075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85981f48e0 con 0x7f8584005bc0 2024-03-29T04:09:26.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.796+0000 7f859f7a8640 1 -- 172.21.15.97:0/1618790075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85980adc00 con 0x7f8584005bc0 2024-03-29T04:09:26.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.796+0000 7f858dffb640 1 -- 172.21.15.97:0/1618790075 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8588045020 con 0x7f8584005bc0 2024-03-29T04:09:26.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.796+0000 7f858dffb640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f857c06c530 0x7f857c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.797+0000 7f859cd43640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f857c06c530 0x7f857c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.797+0000 7f858dffb640 1 -- 172.21.15.97:0/1618790075 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f85880acaa0 con 0x7f8584005bc0 2024-03-29T04:09:26.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.797+0000 7f859cd43640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f857c06c530 0x7f857c06e9f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f859815d480 tx=0x7f8594009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.801+0000 7f858dffb640 1 -- 172.21.15.97:0/1618790075 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f85980adc00 con 0x7f8584005bc0 2024-03-29T04:09:26.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.983+0000 7f859f7a8640 1 -- 172.21.15.97:0/1618790075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8598072ee0 con 0x7f8584005bc0 2024-03-29T04:09:26.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.984+0000 7f858dffb640 1 -- 172.21.15.97:0/1618790075 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16669 (secure 0 0 0) 0x7f8598072ee0 con 0x7f8584005bc0 2024-03-29T04:09:26.964 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:26.964 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":32,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:08:31.615419+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:08:28.403360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T04:08:28.909709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:26.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.986+0000 7f859f7a8640 1 -- 172.21.15.97:0/1618790075 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f857c06c530 msgr2=0x7f857c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.986+0000 7f859f7a8640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f857c06c530 0x7f857c06e9f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f859815d480 tx=0x7f8594009190 comp rx=0 tx=0).stop 2024-03-29T04:09:26.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.986+0000 7f859f7a8640 1 -- 172.21.15.97:0/1618790075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8584005bc0 msgr2=0x7f8598153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.986+0000 7f859f7a8640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8584005bc0 0x7f8598153db0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8588037040 tx=0x7f8588004730 comp rx=0 tx=0).stop 2024-03-29T04:09:26.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.986+0000 7f859f7a8640 1 -- 172.21.15.97:0/1618790075 shutdown_connections 2024-03-29T04:09:26.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.986+0000 7f859f7a8640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f857c06c530 0x7f857c06e9f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.986+0000 7f859f7a8640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8598158e60 0x7f8598155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.986+0000 7f859f7a8640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85981f0a30 0x7f859815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.986+0000 7f859f7a8640 1 --2- 172.21.15.97:0/1618790075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8584005bc0 0x7f8598153db0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.986+0000 7f859f7a8640 1 -- 172.21.15.97:0/1618790075 >> 172.21.15.97:0/1618790075 conn(0x7f859806b510 msgr2=0x7f859806e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.986+0000 7f859f7a8640 1 -- 172.21.15.97:0/1618790075 shutdown_connections 2024-03-29T04:09:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:26.986+0000 7f859f7a8640 1 -- 172.21.15.97:0/1618790075 wait complete. 2024-03-29T04:09:26.978 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:08:28.403360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:08:28.909709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:09:26.980 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:27.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.106+0000 7f12d0628640 1 Processor -- start 2024-03-29T04:09:27.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.106+0000 7f12d0628640 1 -- start start 2024-03-29T04:09:27.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.107+0000 7f12d0628640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f12c809aa30 0x7f12c8162610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.107+0000 7f12d0628640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f12c8162b50 0x7f12c815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.107+0000 7f12d0628640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12c80d19c0 con 0x7f12c80b1980 2024-03-29T04:09:27.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.107+0000 7f12d0628640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f12c8162fa0 con 0x7f12c8162b50 2024-03-29T04:09:27.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.107+0000 7f12d0628640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f12c809aec0 con 0x7f12c809aa30 2024-03-29T04:09:27.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.107+0000 7f12cebc5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f12c8162b50 0x7f12c815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.107+0000 7f12ce3c4640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f12c80b1980 0x7f12c80d12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51366/0 (socket says 172.21.15.97:51366) 2024-03-29T04:09:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.107+0000 7f12ce3c4640 1 -- 172.21.15.97:0/2690551323 learned_addr learned my addr 172.21.15.97:0/2690551323 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.107+0000 7f12cdbc3640 1 --2- 172.21.15.97:0/2690551323 >> v2:172.21.15.154:3300/0 conn(0x7f12c809aa30 0x7f12c8162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.107+0000 7f12cebc5640 1 -- 172.21.15.97:0/2690551323 >> v2:172.21.15.154:3300/0 conn(0x7f12c809aa30 msgr2=0x7f12c8162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.107+0000 7f12cebc5640 1 --2- 172.21.15.97:0/2690551323 >> v2:172.21.15.154:3300/0 conn(0x7f12c809aa30 0x7f12c8162610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.107+0000 7f12cebc5640 1 -- 172.21.15.97:0/2690551323 >> v1:172.21.15.97:6789/0 conn(0x7f12c80b1980 legacy=0x7f12c80d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.108+0000 7f12cebc5640 1 -- 172.21.15.97:0/2690551323 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c809b1a0 con 0x7f12c8162b50 2024-03-29T04:09:27.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.108+0000 7f12cdbc3640 1 --2- 172.21.15.97:0/2690551323 >> v2:172.21.15.154:3300/0 conn(0x7f12c809aa30 0x7f12c8162610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.108+0000 7f12cebc5640 1 --2- 172.21.15.97:0/2690551323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12c8162b50 0x7f12c815e0c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f12c8071a20 tx=0x7f12c002bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa8b59549b1c3c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.108+0000 7f12cd3c2640 1 -- 172.21.15.97:0/2690551323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c0039070 con 0x7f12c8162b50 2024-03-29T04:09:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.108+0000 7f12cd3c2640 1 -- 172.21.15.97:0/2690551323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12c002ca30 con 0x7f12c8162b50 2024-03-29T04:09:27.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.108+0000 7f12cd3c2640 1 -- 172.21.15.97:0/2690551323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c002ce00 con 0x7f12c8162b50 2024-03-29T04:09:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.109+0000 7f12d0628640 1 -- 172.21.15.97:0/2690551323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12c8162b50 msgr2=0x7f12c815e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.109+0000 7f12d0628640 1 --2- 172.21.15.97:0/2690551323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12c8162b50 0x7f12c815e0c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f12c8071a20 tx=0x7f12c002bd00 comp rx=0 tx=0).stop 2024-03-29T04:09:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.109+0000 7f12d0628640 1 -- 172.21.15.97:0/2690551323 shutdown_connections 2024-03-29T04:09:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.109+0000 7f12d0628640 1 --2- 172.21.15.97:0/2690551323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12c8162b50 0x7f12c815e0c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.109+0000 7f12d0628640 1 --2- 172.21.15.97:0/2690551323 >> v2:172.21.15.154:3300/0 conn(0x7f12c809aa30 0x7f12c8162610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.109+0000 7f12d0628640 1 -- 172.21.15.97:0/2690551323 >> 172.21.15.97:0/2690551323 conn(0x7f12c80b3f40 msgr2=0x7f12c80b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.109+0000 7f12d0628640 1 -- 172.21.15.97:0/2690551323 shutdown_connections 2024-03-29T04:09:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.109+0000 7f12d0628640 1 -- 172.21.15.97:0/2690551323 wait complete. 2024-03-29T04:09:27.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.111+0000 7f12d0628640 1 Processor -- start 2024-03-29T04:09:27.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.111+0000 7f12d0628640 1 -- start start 2024-03-29T04:09:27.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12d0628640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12c80b1980 0x7f12c80b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12d0628640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12c8162b50 0x7f12c80b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12d0628640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c81e8340 0x7f12c81e87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12d0628640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f12c809ba40 con 0x7f12c80b1980 2024-03-29T04:09:27.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12d0628640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f12c809b760 con 0x7f12c8162b50 2024-03-29T04:09:27.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12d0628640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f12c815e600 con 0x7f12c81e8340 2024-03-29T04:09:27.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12ce3c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12c80b1980 0x7f12c80b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12ce3c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12c80b1980 0x7f12c80b6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42400/0 (socket says 172.21.15.97:42400) 2024-03-29T04:09:27.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12ce3c4640 1 -- 172.21.15.97:0/3016854874 learned_addr learned my addr 172.21.15.97:0/3016854874 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:27.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12ce3c4640 1 -- 172.21.15.97:0/3016854874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c81e8340 msgr2=0x7f12c81e87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12ce3c4640 1 --2- 172.21.15.97:0/3016854874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c81e8340 0x7f12c81e87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12ce3c4640 1 -- 172.21.15.97:0/3016854874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12c8162b50 msgr2=0x7f12c80b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12ce3c4640 1 --2- 172.21.15.97:0/3016854874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12c8162b50 0x7f12c80b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.112+0000 7f12ce3c4640 1 -- 172.21.15.97:0/3016854874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c80b73e0 con 0x7f12c80b1980 2024-03-29T04:09:27.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.113+0000 7f12ce3c4640 1 --2- 172.21.15.97:0/3016854874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12c80b1980 0x7f12c80b6ea0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f12b800c7b0 tx=0x7f12b800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.113+0000 7f12beffd640 1 -- 172.21.15.97:0/3016854874 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8018070 con 0x7f12c80b1980 2024-03-29T04:09:27.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.113+0000 7f12beffd640 1 -- 172.21.15.97:0/3016854874 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b80072d0 con 0x7f12c80b1980 2024-03-29T04:09:27.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.113+0000 7f12d0628640 1 -- 172.21.15.97:0/3016854874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c81e8d10 con 0x7f12c80b1980 2024-03-29T04:09:27.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.113+0000 7f12beffd640 1 -- 172.21.15.97:0/3016854874 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b80047e0 con 0x7f12c80b1980 2024-03-29T04:09:27.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.113+0000 7f12d0628640 1 -- 172.21.15.97:0/3016854874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c80b66a0 con 0x7f12c80b1980 2024-03-29T04:09:27.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.114+0000 7f12d0628640 1 -- 172.21.15.97:0/3016854874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c80cc810 con 0x7f12c80b1980 2024-03-29T04:09:27.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.114+0000 7f12beffd640 1 -- 172.21.15.97:0/3016854874 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f12b801b070 con 0x7f12c80b1980 2024-03-29T04:09:27.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.115+0000 7f12beffd640 1 --2- 172.21.15.97:0/3016854874 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f12ac06c530 0x7f12ac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.115+0000 7f12cdbc3640 1 --2- 172.21.15.97:0/3016854874 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f12ac06c530 0x7f12ac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.115+0000 7f12beffd640 1 -- 172.21.15.97:0/3016854874 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f12b808aba0 con 0x7f12c80b1980 2024-03-29T04:09:27.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.115+0000 7f12cdbc3640 1 --2- 172.21.15.97:0/3016854874 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f12ac06c530 0x7f12ac06e9f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f12c4006fd0 tx=0x7f12c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.123+0000 7f12beffd640 1 -- 172.21.15.97:0/3016854874 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f12c80cc810 con 0x7f12c80b1980 2024-03-29T04:09:27.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.302+0000 7f12d0628640 1 -- 172.21.15.97:0/3016854874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f12c80a0520 con 0x7f12c80b1980 2024-03-29T04:09:27.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.302+0000 7f12beffd640 1 -- 172.21.15.97:0/3016854874 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) ==== 93+0+10 (secure 0 0 0) 0x7f12b80591c0 con 0x7f12c80b1980 2024-03-29T04:09:27.282 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.304+0000 7f12d0628640 1 -- 172.21.15.97:0/3016854874 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f12ac06c530 msgr2=0x7f12ac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.304+0000 7f12d0628640 1 --2- 172.21.15.97:0/3016854874 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f12ac06c530 0x7f12ac06e9f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f12c4006fd0 tx=0x7f12c4008040 comp rx=0 tx=0).stop 2024-03-29T04:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.304+0000 7f12d0628640 1 -- 172.21.15.97:0/3016854874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12c80b1980 msgr2=0x7f12c80b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.304+0000 7f12d0628640 1 --2- 172.21.15.97:0/3016854874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12c80b1980 0x7f12c80b6ea0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f12b800c7b0 tx=0x7f12b800cc70 comp rx=0 tx=0).stop 2024-03-29T04:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.304+0000 7f12d0628640 1 -- 172.21.15.97:0/3016854874 shutdown_connections 2024-03-29T04:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.304+0000 7f12d0628640 1 --2- 172.21.15.97:0/3016854874 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f12ac06c530 0x7f12ac06e9f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.304+0000 7f12d0628640 1 --2- 172.21.15.97:0/3016854874 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f12c81e8340 0x7f12c81e87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.304+0000 7f12d0628640 1 --2- 172.21.15.97:0/3016854874 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12c8162b50 0x7f12c80b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.304+0000 7f12d0628640 1 --2- 172.21.15.97:0/3016854874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12c80b1980 0x7f12c80b6ea0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.304+0000 7f12d0628640 1 -- 172.21.15.97:0/3016854874 >> 172.21.15.97:0/3016854874 conn(0x7f12c80b3f40 msgr2=0x7f12c80b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.304+0000 7f12d0628640 1 -- 172.21.15.97:0/3016854874 shutdown_connections 2024-03-29T04:09:27.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.304+0000 7f12d0628640 1 -- 172.21.15.97:0/3016854874 wait complete. 2024-03-29T04:09:27.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:09:27.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.420+0000 7f1b7f22a640 1 Processor -- start 2024-03-29T04:09:27.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.420+0000 7f1b7f22a640 1 -- start start 2024-03-29T04:09:27.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.420+0000 7f1b7f22a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b7815e920 0x7f1b780d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.420+0000 7f1b7f22a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b780d16a0 0x7f1b780d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.420+0000 7f1b7f22a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b780d3780 con 0x7f1b780cfde0 2024-03-29T04:09:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.420+0000 7f1b7f22a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f1b7815ed00 con 0x7f1b7815e920 2024-03-29T04:09:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.420+0000 7f1b7f22a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1b780d21e0 con 0x7f1b780d16a0 2024-03-29T04:09:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.421+0000 7f1b7cfc6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b7815e920 0x7f1b780d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.421+0000 7f1b77fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1b780cfde0 0x7f1b780d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51382/0 (socket says 172.21.15.97:51382) 2024-03-29T04:09:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.421+0000 7f1b77fff640 1 -- 172.21.15.97:0/2582943446 learned_addr learned my addr 172.21.15.97:0/2582943446 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.421+0000 7f1b7d7c7640 1 --2- 172.21.15.97:0/2582943446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b780d16a0 0x7f1b780d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.421+0000 7f1b7cfc6640 1 -- 172.21.15.97:0/2582943446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b780d16a0 msgr2=0x7f1b780d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.421+0000 7f1b7cfc6640 1 --2- 172.21.15.97:0/2582943446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b780d16a0 0x7f1b780d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.421+0000 7f1b7cfc6640 1 -- 172.21.15.97:0/2582943446 >> v1:172.21.15.97:6789/0 conn(0x7f1b780cfde0 legacy=0x7f1b780d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.421+0000 7f1b7cfc6640 1 -- 172.21.15.97:0/2582943446 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b780d2480 con 0x7f1b7815e920 2024-03-29T04:09:27.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.421+0000 7f1b777fe640 1 -- 172.21.15.97:0/2582943446 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 767604428 0 0) 0x7f1b64001110 con 0x7f1b780cfde0 2024-03-29T04:09:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.422+0000 7f1b7cfc6640 1 --2- 172.21.15.97:0/2582943446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7815e920 0x7f1b780d3240 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0097a0 tx=0x7f1b6c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6265b2d67beeb9d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.422+0000 7f1b777fe640 1 -- 172.21.15.97:0/2582943446 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c035020 con 0x7f1b7815e920 2024-03-29T04:09:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.422+0000 7f1b777fe640 1 -- 172.21.15.97:0/2582943446 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c031830 con 0x7f1b7815e920 2024-03-29T04:09:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.422+0000 7f1b777fe640 1 -- 172.21.15.97:0/2582943446 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c031c00 con 0x7f1b7815e920 2024-03-29T04:09:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.422+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2582943446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7815e920 msgr2=0x7f1b780d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.422+0000 7f1b7f22a640 1 --2- 172.21.15.97:0/2582943446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7815e920 0x7f1b780d3240 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0097a0 tx=0x7f1b6c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.422+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2582943446 shutdown_connections 2024-03-29T04:09:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.422+0000 7f1b7f22a640 1 --2- 172.21.15.97:0/2582943446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b780d16a0 0x7f1b780d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.422+0000 7f1b7f22a640 1 --2- 172.21.15.97:0/2582943446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7815e920 0x7f1b780d3240 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.422+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2582943446 >> 172.21.15.97:0/2582943446 conn(0x7f1b7808bfc0 msgr2=0x7f1b780d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:27.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.422+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2582943446 shutdown_connections 2024-03-29T04:09:27.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.422+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2582943446 wait complete. 2024-03-29T04:09:27.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.424+0000 7f1b7f22a640 1 Processor -- start 2024-03-29T04:09:27.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.424+0000 7f1b7f22a640 1 -- start start 2024-03-29T04:09:27.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7f22a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b780cfde0 0x7f1b780c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7f22a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780d16a0 0x7f1b780b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7f22a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7815e920 0x7f1b780b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7f22a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1b780d2d20 con 0x7f1b7815e920 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7f22a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1b780d2a40 con 0x7f1b780d16a0 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7f22a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1b780d4ef0 con 0x7f1b780cfde0 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7d7c7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7815e920 0x7f1b780b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7d7c7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7815e920 0x7f1b780b6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42424/0 (socket says 172.21.15.97:42424) 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7d7c7640 1 -- 172.21.15.97:0/2989940580 learned_addr learned my addr 172.21.15.97:0/2989940580 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7cfc6640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b780cfde0 0x7f1b780c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b77fff640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780d16a0 0x7f1b780b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7d7c7640 1 -- 172.21.15.97:0/2989940580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b780cfde0 msgr2=0x7f1b780c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7d7c7640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b780cfde0 0x7f1b780c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7d7c7640 1 -- 172.21.15.97:0/2989940580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780d16a0 msgr2=0x7f1b780b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7d7c7640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780d16a0 0x7f1b780b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7d7c7640 1 -- 172.21.15.97:0/2989940580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b780ca280 con 0x7f1b7815e920 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7d7c7640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7815e920 0x7f1b780b6780 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1b70007e60 tx=0x7f1b7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b757fa640 1 -- 172.21.15.97:0/2989940580 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70014420 con 0x7f1b7815e920 2024-03-29T04:09:27.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b757fa640 1 -- 172.21.15.97:0/2989940580 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b70019070 con 0x7f1b7815e920 2024-03-29T04:09:27.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2989940580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b780ca560 con 0x7f1b7815e920 2024-03-29T04:09:27.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b757fa640 1 -- 172.21.15.97:0/2989940580 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70014820 con 0x7f1b7815e920 2024-03-29T04:09:27.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.426+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2989940580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b780cab20 con 0x7f1b7815e920 2024-03-29T04:09:27.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.428+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2989940580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b781563e0 con 0x7f1b7815e920 2024-03-29T04:09:27.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.428+0000 7f1b757fa640 1 -- 172.21.15.97:0/2989940580 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1b700149c0 con 0x7f1b7815e920 2024-03-29T04:09:27.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.428+0000 7f1b757fa640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b5c06c530 0x7f1b5c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.428+0000 7f1b757fa640 1 -- 172.21.15.97:0/2989940580 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f1b7008bd60 con 0x7f1b7815e920 2024-03-29T04:09:27.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.428+0000 7f1b7cfc6640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b5c06c530 0x7f1b5c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.429+0000 7f1b7cfc6640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b5c06c530 0x7f1b5c06e9f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c036000 tx=0x7f1b6c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.433+0000 7f1b757fa640 1 -- 172.21.15.97:0/2989940580 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1b781563e0 con 0x7f1b7815e920 2024-03-29T04:09:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.613+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2989940580 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1b780a2560 con 0x7f1b7815e920 2024-03-29T04:09:27.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.613+0000 7f1b757fa640 1 -- 172.21.15.97:0/2989940580 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) ==== 104+0+11 (secure 0 0 0) 0x7f1b70059dc0 con 0x7f1b7815e920 2024-03-29T04:09:27.593 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:09:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.615+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2989940580 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b5c06c530 msgr2=0x7f1b5c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.615+0000 7f1b7f22a640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b5c06c530 0x7f1b5c06e9f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c036000 tx=0x7f1b6c03d040 comp rx=0 tx=0).stop 2024-03-29T04:09:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.615+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2989940580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7815e920 msgr2=0x7f1b780b6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.615+0000 7f1b7f22a640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7815e920 0x7f1b780b6780 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1b70007e60 tx=0x7f1b7000e4b0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.615+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2989940580 shutdown_connections 2024-03-29T04:09:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.615+0000 7f1b7f22a640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b5c06c530 0x7f1b5c06e9f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.615+0000 7f1b7f22a640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b7815e920 0x7f1b780b6780 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.615+0000 7f1b7f22a640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780d16a0 0x7f1b780b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.615+0000 7f1b7f22a640 1 --2- 172.21.15.97:0/2989940580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b780cfde0 0x7f1b780c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.615+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2989940580 >> 172.21.15.97:0/2989940580 conn(0x7f1b7808bfc0 msgr2=0x7f1b7806dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:27.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.615+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2989940580 shutdown_connections 2024-03-29T04:09:27.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.615+0000 7f1b7f22a640 1 -- 172.21.15.97:0/2989940580 wait complete. 2024-03-29T04:09:27.606 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:09:27.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.731+0000 7fd74ba70640 1 Processor -- start 2024-03-29T04:09:27.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.731+0000 7fd74ba70640 1 -- start start 2024-03-29T04:09:27.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.732+0000 7fd74ba70640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd74415e8b0 0x7fd7440d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.732+0000 7fd74ba70640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd7440d0e70 0x7fd7440d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.732+0000 7fd74ba70640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7440d13e0 con 0x7fd74415e4d0 2024-03-29T04:09:27.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.732+0000 7fd74ba70640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fd7440cf6b0 con 0x7fd7440d0e70 2024-03-29T04:09:27.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.732+0000 7fd74ba70640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd7440d16c0 con 0x7fd74415e8b0 2024-03-29T04:09:27.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.732+0000 7fd74a00d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd7440d0e70 0x7fd7440d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.732+0000 7fd74a00d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd7440d0e70 0x7fd7440d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42426/0 (socket says 172.21.15.97:42426) 2024-03-29T04:09:27.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.732+0000 7fd74a00d640 1 -- 172.21.15.97:0/2623046610 learned_addr learned my addr 172.21.15.97:0/2623046610 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:27.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.732+0000 7fd74900b640 1 --2- 172.21.15.97:0/2623046610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd74415e8b0 0x7fd7440d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.733+0000 7fd74a00d640 1 -- 172.21.15.97:0/2623046610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd74415e8b0 msgr2=0x7fd7440d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:27.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.733+0000 7fd74a00d640 1 --2- 172.21.15.97:0/2623046610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd74415e8b0 0x7fd7440d2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.733+0000 7fd74a00d640 1 -- 172.21.15.97:0/2623046610 >> v1:172.21.15.154:6789/0 conn(0x7fd74415e4d0 legacy=0x7fd74406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.733+0000 7fd74a00d640 1 -- 172.21.15.97:0/2623046610 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7440d19a0 con 0x7fd7440d0e70 2024-03-29T04:09:27.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.733+0000 7fd74900b640 1 --2- 172.21.15.97:0/2623046610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd74415e8b0 0x7fd7440d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:27.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.734+0000 7fd74a00d640 1 --2- 172.21.15.97:0/2623046610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7440d0e70 0x7fd7440d2bf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009e50 tx=0x7fd73c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd247a125705900d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.734+0000 7fd74880a640 1 -- 172.21.15.97:0/2623046610 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c02cb00 con 0x7fd7440d0e70 2024-03-29T04:09:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.734+0000 7fd74880a640 1 -- 172.21.15.97:0/2623046610 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c02cca0 con 0x7fd7440d0e70 2024-03-29T04:09:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.734+0000 7fd74880a640 1 -- 172.21.15.97:0/2623046610 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c034610 con 0x7fd7440d0e70 2024-03-29T04:09:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.734+0000 7fd74ba70640 1 -- 172.21.15.97:0/2623046610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7440d0e70 msgr2=0x7fd7440d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.734+0000 7fd74ba70640 1 --2- 172.21.15.97:0/2623046610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7440d0e70 0x7fd7440d2bf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009e50 tx=0x7fd73c02bf60 comp rx=0 tx=0).stop 2024-03-29T04:09:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.734+0000 7fd74ba70640 1 -- 172.21.15.97:0/2623046610 shutdown_connections 2024-03-29T04:09:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.734+0000 7fd74ba70640 1 --2- 172.21.15.97:0/2623046610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7440d0e70 0x7fd7440d2bf0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.734+0000 7fd74ba70640 1 --2- 172.21.15.97:0/2623046610 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd74415e8b0 0x7fd7440d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.734+0000 7fd74ba70640 1 -- 172.21.15.97:0/2623046610 >> 172.21.15.97:0/2623046610 conn(0x7fd74408bfc0 msgr2=0x7fd7440d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.735+0000 7fd74ba70640 1 -- 172.21.15.97:0/2623046610 shutdown_connections 2024-03-29T04:09:27.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.735+0000 7fd74ba70640 1 -- 172.21.15.97:0/2623046610 wait complete. 2024-03-29T04:09:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.736+0000 7fd74ba70640 1 Processor -- start 2024-03-29T04:09:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.736+0000 7fd74ba70640 1 -- start start 2024-03-29T04:09:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.736+0000 7fd74ba70640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7440d0e70 0x7fd7440ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.736+0000 7fd74ba70640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd74415e4d0 0x7fd7440cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.736+0000 7fd74ba70640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74415e8b0 0x7fd7440ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.736+0000 7fd74ba70640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7440d2890 con 0x7fd74415e8b0 2024-03-29T04:09:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.736+0000 7fd74ba70640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd7440d3130 con 0x7fd74415e4d0 2024-03-29T04:09:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.736+0000 7fd74ba70640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd7440cab10 con 0x7fd7440d0e70 2024-03-29T04:09:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd74a00d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74415e8b0 0x7fd7440ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd74a00d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74415e8b0 0x7fd7440ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42430/0 (socket says 172.21.15.97:42430) 2024-03-29T04:09:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd74a00d640 1 -- 172.21.15.97:0/1797008064 learned_addr learned my addr 172.21.15.97:0/1797008064 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd74980c640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7440d0e70 0x7fd7440ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd74900b640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd74415e4d0 0x7fd7440cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd74a00d640 1 -- 172.21.15.97:0/1797008064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7440d0e70 msgr2=0x7fd7440ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd74a00d640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7440d0e70 0x7fd7440ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd74a00d640 1 -- 172.21.15.97:0/1797008064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd74415e4d0 msgr2=0x7fd7440cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd74a00d640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd74415e4d0 0x7fd7440cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd74a00d640 1 -- 172.21.15.97:0/1797008064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7440c9530 con 0x7fd74415e8b0 2024-03-29T04:09:27.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd74a00d640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74415e8b0 0x7fd7440ce290 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009f70 tx=0x7fd73c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd73a7fc640 1 -- 172.21.15.97:0/1797008064 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c043020 con 0x7fd74415e8b0 2024-03-29T04:09:27.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd73a7fc640 1 -- 172.21.15.97:0/1797008064 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c02ce50 con 0x7fd74415e8b0 2024-03-29T04:09:27.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd74ba70640 1 -- 172.21.15.97:0/1797008064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7440c9810 con 0x7fd74415e8b0 2024-03-29T04:09:27.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd73a7fc640 1 -- 172.21.15.97:0/1797008064 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c009510 con 0x7fd74415e8b0 2024-03-29T04:09:27.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.737+0000 7fd74ba70640 1 -- 172.21.15.97:0/1797008064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7440c9dd0 con 0x7fd74415e8b0 2024-03-29T04:09:27.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.738+0000 7fd74ba70640 1 -- 172.21.15.97:0/1797008064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7440a2500 con 0x7fd74415e8b0 2024-03-29T04:09:27.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.739+0000 7fd73a7fc640 1 -- 172.21.15.97:0/1797008064 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd73c005d10 con 0x7fd74415e8b0 2024-03-29T04:09:27.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.739+0000 7fd73a7fc640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd72806c530 0x7fd72806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.739+0000 7fd73a7fc640 1 -- 172.21.15.97:0/1797008064 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd73c0adde0 con 0x7fd74415e8b0 2024-03-29T04:09:27.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.739+0000 7fd74980c640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd72806c530 0x7fd72806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.740+0000 7fd74980c640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd72806c530 0x7fd72806e9f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd734000f60 tx=0x7fd734009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.743+0000 7fd73a7fc640 1 -- 172.21.15.97:0/1797008064 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd73c07be40 con 0x7fd74415e8b0 2024-03-29T04:09:27.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.923+0000 7fd74ba70640 1 -- 172.21.15.97:0/1797008064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd74404f9d0 con 0x7fd74415e8b0 2024-03-29T04:09:27.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.923+0000 7fd73a7fc640 1 -- 172.21.15.97:0/1797008064 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) ==== 100+0+11 (secure 0 0 0) 0x7fd73c080140 con 0x7fd74415e8b0 2024-03-29T04:09:27.903 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:09:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.925+0000 7fd74ba70640 1 -- 172.21.15.97:0/1797008064 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd72806c530 msgr2=0x7fd72806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.925+0000 7fd74ba70640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd72806c530 0x7fd72806e9f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd734000f60 tx=0x7fd734009040 comp rx=0 tx=0).stop 2024-03-29T04:09:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.925+0000 7fd74ba70640 1 -- 172.21.15.97:0/1797008064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74415e8b0 msgr2=0x7fd7440ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.925+0000 7fd74ba70640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74415e8b0 0x7fd7440ce290 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009f70 tx=0x7fd73c004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.925+0000 7fd74ba70640 1 -- 172.21.15.97:0/1797008064 shutdown_connections 2024-03-29T04:09:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.925+0000 7fd74ba70640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd72806c530 0x7fd72806e9f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.925+0000 7fd74ba70640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd74415e8b0 0x7fd7440ce290 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.925+0000 7fd74ba70640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd74415e4d0 0x7fd7440cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.925+0000 7fd74ba70640 1 --2- 172.21.15.97:0/1797008064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7440d0e70 0x7fd7440ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.925+0000 7fd74ba70640 1 -- 172.21.15.97:0/1797008064 >> 172.21.15.97:0/1797008064 conn(0x7fd74408bfc0 msgr2=0x7fd74415c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.925+0000 7fd74ba70640 1 -- 172.21.15.97:0/1797008064 shutdown_connections 2024-03-29T04:09:27.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:27.925+0000 7fd74ba70640 1 -- 172.21.15.97:0/1797008064 wait complete. 2024-03-29T04:09:27.916 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:09:28.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.038+0000 7f630fd24640 1 Processor -- start 2024-03-29T04:09:28.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.038+0000 7f630fd24640 1 -- start start 2024-03-29T04:09:28.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.040+0000 7f630fd24640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f630815c740 0x7f63080cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.040+0000 7f630fd24640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f63080cceb0 0x7f63080d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.040+0000 7f630fd24640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63080d1310 con 0x7f63080d0960 2024-03-29T04:09:28.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.040+0000 7f630fd24640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f63080cd310 con 0x7f63080cceb0 2024-03-29T04:09:28.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.040+0000 7f630fd24640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f63080d0dd0 con 0x7f630815c740 2024-03-29T04:09:28.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.040+0000 7f630d2bf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f63080cceb0 0x7f63080d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.040+0000 7f630d2bf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f63080cceb0 0x7f63080d0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42432/0 (socket says 172.21.15.97:42432) 2024-03-29T04:09:28.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.040+0000 7f630d2bf640 1 -- 172.21.15.97:0/4058140376 learned_addr learned my addr 172.21.15.97:0/4058140376 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:28.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.040+0000 7f630dac0640 1 --2- 172.21.15.97:0/4058140376 >> v2:172.21.15.154:3300/0 conn(0x7f630815c740 0x7f63080cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.040+0000 7f630d2bf640 1 -- 172.21.15.97:0/4058140376 >> v2:172.21.15.154:3300/0 conn(0x7f630815c740 msgr2=0x7f63080cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:28.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.040+0000 7f630d2bf640 1 --2- 172.21.15.97:0/4058140376 >> v2:172.21.15.154:3300/0 conn(0x7f630815c740 0x7f63080cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.040+0000 7f630d2bf640 1 -- 172.21.15.97:0/4058140376 >> v1:172.21.15.154:6789/0 conn(0x7f63080d0960 legacy=0x7f6308160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.040+0000 7f630d2bf640 1 -- 172.21.15.97:0/4058140376 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6308070a20 con 0x7f63080cceb0 2024-03-29T04:09:28.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.041+0000 7f630d2bf640 1 --2- 172.21.15.97:0/4058140376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63080cceb0 0x7f63080d0420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63080734c0 tx=0x7f62f802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f403d87b80e792a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.042+0000 7f630cabe640 1 -- 172.21.15.97:0/4058140376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8034070 con 0x7f63080cceb0 2024-03-29T04:09:28.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.042+0000 7f630cabe640 1 -- 172.21.15.97:0/4058140376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62f8030860 con 0x7f63080cceb0 2024-03-29T04:09:28.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.042+0000 7f630cabe640 1 -- 172.21.15.97:0/4058140376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8030c30 con 0x7f63080cceb0 2024-03-29T04:09:28.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.042+0000 7f630fd24640 1 -- 172.21.15.97:0/4058140376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63080cceb0 msgr2=0x7f63080d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.042+0000 7f630fd24640 1 --2- 172.21.15.97:0/4058140376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63080cceb0 0x7f63080d0420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63080734c0 tx=0x7f62f802b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.042+0000 7f630fd24640 1 -- 172.21.15.97:0/4058140376 shutdown_connections 2024-03-29T04:09:28.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.042+0000 7f630fd24640 1 --2- 172.21.15.97:0/4058140376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63080cceb0 0x7f63080d0420 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.042+0000 7f630fd24640 1 --2- 172.21.15.97:0/4058140376 >> v2:172.21.15.154:3300/0 conn(0x7f630815c740 0x7f63080cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.042+0000 7f630fd24640 1 -- 172.21.15.97:0/4058140376 >> 172.21.15.97:0/4058140376 conn(0x7f63080b3e90 msgr2=0x7f63080b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:28.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.042+0000 7f630fd24640 1 -- 172.21.15.97:0/4058140376 shutdown_connections 2024-03-29T04:09:28.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.042+0000 7f630fd24640 1 -- 172.21.15.97:0/4058140376 wait complete. 2024-03-29T04:09:28.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.043+0000 7f630fd24640 1 Processor -- start 2024-03-29T04:09:28.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.043+0000 7f630fd24640 1 -- start start 2024-03-29T04:09:28.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630fd24640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63080cceb0 0x7f630814c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630fd24640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63080d0960 0x7f63080b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630fd24640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f630815c740 0x7f63080b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630fd24640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f630809af20 con 0x7f630815c740 2024-03-29T04:09:28.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630fd24640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f630809b850 con 0x7f63080d0960 2024-03-29T04:09:28.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630fd24640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f630809b200 con 0x7f63080cceb0 2024-03-29T04:09:28.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630e2c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f630815c740 0x7f63080b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630e2c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f630815c740 0x7f63080b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42448/0 (socket says 172.21.15.97:42448) 2024-03-29T04:09:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630e2c1640 1 -- 172.21.15.97:0/2786858253 learned_addr learned my addr 172.21.15.97:0/2786858253 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630d2bf640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63080d0960 0x7f63080b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630dac0640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63080cceb0 0x7f630814c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630e2c1640 1 -- 172.21.15.97:0/2786858253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63080cceb0 msgr2=0x7f630814c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630e2c1640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63080cceb0 0x7f630814c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630e2c1640 1 -- 172.21.15.97:0/2786858253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63080d0960 msgr2=0x7f63080b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630e2c1640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63080d0960 0x7f63080b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.044+0000 7f630e2c1640 1 -- 172.21.15.97:0/2786858253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630815ece0 con 0x7f630815c740 2024-03-29T04:09:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.045+0000 7f630e2c1640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f630815c740 0x7f63080b5590 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f630000c6e0 tx=0x7f630000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.045+0000 7f62fe7fc640 1 -- 172.21.15.97:0/2786858253 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300003b70 con 0x7f630815c740 2024-03-29T04:09:28.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.045+0000 7f62fe7fc640 1 -- 172.21.15.97:0/2786858253 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6300003d30 con 0x7f630815c740 2024-03-29T04:09:28.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.045+0000 7f630fd24640 1 -- 172.21.15.97:0/2786858253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63080c9d00 con 0x7f630815c740 2024-03-29T04:09:28.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.045+0000 7f62fe7fc640 1 -- 172.21.15.97:0/2786858253 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300005310 con 0x7f630815c740 2024-03-29T04:09:28.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.045+0000 7f630fd24640 1 -- 172.21.15.97:0/2786858253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630815efc0 con 0x7f630815c740 2024-03-29T04:09:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.046+0000 7f630fd24640 1 -- 172.21.15.97:0/2786858253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6308068d50 con 0x7f630815c740 2024-03-29T04:09:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.046+0000 7f62fe7fc640 1 -- 172.21.15.97:0/2786858253 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f63000054b0 con 0x7f630815c740 2024-03-29T04:09:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.047+0000 7f62fe7fc640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f62ec06c530 0x7f62ec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.047+0000 7f62fe7fc640 1 -- 172.21.15.97:0/2786858253 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f630008b3f0 con 0x7f630815c740 2024-03-29T04:09:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.047+0000 7f630dac0640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f62ec06c530 0x7f62ec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.047+0000 7f630dac0640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f62ec06c530 0x7f62ec06e9f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6304009b40 tx=0x7f6304009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.051+0000 7f62fe7fc640 1 -- 172.21.15.97:0/2786858253 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f63000593c0 con 0x7f630815c740 2024-03-29T04:09:28.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.236+0000 7f630fd24640 1 -- 172.21.15.97:0/2786858253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f63080cfb00 con 0x7f630815c740 2024-03-29T04:09:28.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.237+0000 7f62fe7fc640 1 -- 172.21.15.97:0/2786858253 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7f630005d6c0 con 0x7f630815c740 2024-03-29T04:09:28.216 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-03-29T04:09:28.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.239+0000 7f630fd24640 1 -- 172.21.15.97:0/2786858253 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f62ec06c530 msgr2=0x7f62ec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.239+0000 7f630fd24640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f62ec06c530 0x7f62ec06e9f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6304009b40 tx=0x7f6304009190 comp rx=0 tx=0).stop 2024-03-29T04:09:28.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.239+0000 7f630fd24640 1 -- 172.21.15.97:0/2786858253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f630815c740 msgr2=0x7f63080b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.239+0000 7f630fd24640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f630815c740 0x7f63080b5590 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f630000c6e0 tx=0x7f630000cba0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.239+0000 7f630fd24640 1 -- 172.21.15.97:0/2786858253 shutdown_connections 2024-03-29T04:09:28.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.239+0000 7f630fd24640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f62ec06c530 0x7f62ec06e9f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.239+0000 7f630fd24640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f630815c740 0x7f63080b5590 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.239+0000 7f630fd24640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63080d0960 0x7f63080b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.239+0000 7f630fd24640 1 --2- 172.21.15.97:0/2786858253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f63080cceb0 0x7f630814c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.239+0000 7f630fd24640 1 -- 172.21.15.97:0/2786858253 >> 172.21.15.97:0/2786858253 conn(0x7f63080b3e90 msgr2=0x7f63080cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:28.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.239+0000 7f630fd24640 1 -- 172.21.15.97:0/2786858253 shutdown_connections 2024-03-29T04:09:28.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.239+0000 7f630fd24640 1 -- 172.21.15.97:0/2786858253 wait complete. 2024-03-29T04:09:28.229 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-29T04:09:28.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:28.229 DEBUG:teuthology.orchestra.run.smithi154:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-29T04:09:28.265 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T04:09:28.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:28.265 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T04:09:28.323 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:09:28.323 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:09:28.323 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:28.338 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056369,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18829 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T04:09:28.339 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:09:28.339 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:09:28.339 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:28.380 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18611 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T04:09:28.380 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.501+0000 7f0d530c5640 1 Processor -- start 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.501+0000 7f0d530c5640 1 -- start start 2024-03-29T04:09:28.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.501+0000 7f0d530c5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d4c15e930 0x7f0d4c0cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.501+0000 7f0d530c5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0d4c0d3310 0x7f0d4c06cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.501+0000 7f0d530c5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d4c0cfb30 con 0x7f0d4c0cf670 2024-03-29T04:09:28.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.501+0000 7f0d530c5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f0d4c06c780 con 0x7f0d4c0d3310 2024-03-29T04:09:28.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.501+0000 7f0d530c5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0d4c0cfe10 con 0x7f0d4c15e930 2024-03-29T04:09:28.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4bfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0d4c0cf670 0x7f0d4c0d2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51398/0 (socket says 172.21.15.97:51398) 2024-03-29T04:09:28.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4bfff640 1 -- 172.21.15.97:0/4062406633 learned_addr learned my addr 172.21.15.97:0/4062406633 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d50e61640 1 --2- 172.21.15.97:0/4062406633 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d4c15e930 0x7f0d4c0cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4119564650 0 0) 0x7f0d4c0cfb30 con 0x7f0d4c0cf670 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d51662640 1 --2- 172.21.15.97:0/4062406633 >> v2:172.21.15.154:3300/0 conn(0x7f0d4c0d3310 0x7f0d4c06cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d3c003520 con 0x7f0d4c0cf670 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3734602189 0 0) 0x7f0d3c003520 con 0x7f0d4c0cf670 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d4c15e930 msgr2=0x7f0d4c0cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 --2- 172.21.15.97:0/4062406633 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d4c15e930 0x7f0d4c0cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 >> v2:172.21.15.154:3300/0 conn(0x7f0d4c0d3310 msgr2=0x7f0d4c06cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 --2- 172.21.15.97:0/4062406633 >> v2:172.21.15.154:3300/0 conn(0x7f0d4c0d3310 0x7f0d4c06cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c0d00f0 con 0x7f0d4c0cf670 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f0d38001670 con 0x7f0d4c0cf670 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 >> v1:172.21.15.97:6789/0 conn(0x7f0d4c0cf670 legacy=0x7f0d4c0d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 --2- 172.21.15.97:0/4062406633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d3c0039b0 0x7f0d3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 --2- 172.21.15.97:0/4062406633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d3c004800 0x7f0d3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 --2- 172.21.15.97:0/4062406633 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d4c15e930 0x7f0d3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c06c780 con 0x7f0d3c0039b0 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c0cfe10 con 0x7f0d3c004800 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3c00b9c0 con 0x7f0d4c15e930 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d50e61640 1 --2- 172.21.15.97:0/4062406633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d3c0039b0 0x7f0d3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.502+0000 7f0d4bfff640 1 --2- 172.21.15.97:0/4062406633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d3c004800 0x7f0d3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.503+0000 7f0d51662640 1 --2- 172.21.15.97:0/4062406633 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d4c15e930 0x7f0d3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.503+0000 7f0d50e61640 1 -- 172.21.15.97:0/4062406633 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d4c15e930 msgr2=0x7f0d3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.503+0000 7f0d50e61640 1 --2- 172.21.15.97:0/4062406633 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d4c15e930 0x7f0d3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.503+0000 7f0d50e61640 1 -- 172.21.15.97:0/4062406633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d3c004800 msgr2=0x7f0d3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.503+0000 7f0d50e61640 1 --2- 172.21.15.97:0/4062406633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d3c004800 0x7f0d3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.503+0000 7f0d50e61640 1 -- 172.21.15.97:0/4062406633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c003520 con 0x7f0d3c0039b0 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.503+0000 7f0d4bfff640 1 --2- 172.21.15.97:0/4062406633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d3c004800 0x7f0d3c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:28.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.503+0000 7f0d50e61640 1 --2- 172.21.15.97:0/4062406633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d3c0039b0 0x7f0d3c003dd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009af0 tx=0x7f0d3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b51d1b5e42f996 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.503+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40039070 con 0x7f0d3c0039b0 2024-03-29T04:09:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.503+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4002c6b0 con 0x7f0d3c0039b0 2024-03-29T04:09:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.503+0000 7f0d4b7fe640 1 -- 172.21.15.97:0/4062406633 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40009690 con 0x7f0d3c0039b0 2024-03-29T04:09:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.504+0000 7f0d530c5640 1 -- 172.21.15.97:0/4062406633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d3c0039b0 msgr2=0x7f0d3c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.504+0000 7f0d530c5640 1 --2- 172.21.15.97:0/4062406633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d3c0039b0 0x7f0d3c003dd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009af0 tx=0x7f0d3c005830 comp rx=0 tx=0).stop 2024-03-29T04:09:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.504+0000 7f0d51662640 1 -- 172.21.15.97:0/4062406633 reap_dead start 2024-03-29T04:09:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.504+0000 7f0d530c5640 1 -- 172.21.15.97:0/4062406633 shutdown_connections 2024-03-29T04:09:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.504+0000 7f0d530c5640 1 -- 172.21.15.97:0/4062406633 >> 172.21.15.97:0/4062406633 conn(0x7f0d4c08bfc0 msgr2=0x7f0d4c0d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.504+0000 7f0d530c5640 1 -- 172.21.15.97:0/4062406633 shutdown_connections 2024-03-29T04:09:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.504+0000 7f0d530c5640 1 -- 172.21.15.97:0/4062406633 wait complete. 2024-03-29T04:09:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.505+0000 7f0d530c5640 1 Processor -- start 2024-03-29T04:09:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.505+0000 7f0d530c5640 1 -- start start 2024-03-29T04:09:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d530c5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3c0039b0 0x7f0d4c160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d530c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4c15c210 0x7f0d4c161170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d530c5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d4c1f4630 0x7f0d4c1f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d530c5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c15ee90 con 0x7f0d4c15c210 2024-03-29T04:09:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d530c5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c15ed10 con 0x7f0d4c1f4630 2024-03-29T04:09:28.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d530c5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0d4c0b3c50 con 0x7f0d3c0039b0 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d4bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4c15c210 0x7f0d4c161170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d4bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4c15c210 0x7f0d4c161170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42452/0 (socket says 172.21.15.97:42452) 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d4bfff640 1 -- 172.21.15.97:0/1081358008 learned_addr learned my addr 172.21.15.97:0/1081358008 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d50e61640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3c0039b0 0x7f0d4c160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d51662640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d4c1f4630 0x7f0d4c1f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d4bfff640 1 -- 172.21.15.97:0/1081358008 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3c0039b0 msgr2=0x7f0d4c160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d4bfff640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3c0039b0 0x7f0d4c160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d4bfff640 1 -- 172.21.15.97:0/1081358008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d4c1f4630 msgr2=0x7f0d4c1f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d4bfff640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d4c1f4630 0x7f0d4c1f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d4bfff640 1 -- 172.21.15.97:0/1081358008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c1600d0 con 0x7f0d4c15c210 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.506+0000 7f0d50e61640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3c0039b0 0x7f0d4c160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.507+0000 7f0d4bfff640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4c15c210 0x7f0d4c161170 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800e7b0 tx=0x7f0d3800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.507+0000 7f0d497fa640 1 -- 172.21.15.97:0/1081358008 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3800c870 con 0x7f0d4c15c210 2024-03-29T04:09:28.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.507+0000 7f0d497fa640 1 -- 172.21.15.97:0/1081358008 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d38018070 con 0x7f0d4c15c210 2024-03-29T04:09:28.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.507+0000 7f0d497fa640 1 -- 172.21.15.97:0/1081358008 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3800cc30 con 0x7f0d4c15c210 2024-03-29T04:09:28.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.507+0000 7f0d530c5640 1 -- 172.21.15.97:0/1081358008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c15f070 con 0x7f0d4c15c210 2024-03-29T04:09:28.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.507+0000 7f0d530c5640 1 -- 172.21.15.97:0/1081358008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c15f610 con 0x7f0d4c15c210 2024-03-29T04:09:28.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.508+0000 7f0d530c5640 1 -- 172.21.15.97:0/1081358008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d4c068640 con 0x7f0d4c15c210 2024-03-29T04:09:28.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.508+0000 7f0d497fa640 1 -- 172.21.15.97:0/1081358008 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0d38013420 con 0x7f0d4c15c210 2024-03-29T04:09:28.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.509+0000 7f0d497fa640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0d3006c560 0x7f0d3006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.509+0000 7f0d50e61640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0d3006c560 0x7f0d3006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.509+0000 7f0d497fa640 1 -- 172.21.15.97:0/1081358008 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f0d3808b2a0 con 0x7f0d4c15c210 2024-03-29T04:09:28.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.509+0000 7f0d50e61640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0d3006c560 0x7f0d3006ea20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0d40036000 tx=0x7f0d40030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.513+0000 7f0d497fa640 1 -- 172.21.15.97:0/1081358008 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0d4c1600d0 con 0x7f0d4c15c210 2024-03-29T04:09:28.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.730+0000 7f0d530c5640 1 -- 172.21.15.97:0/1081358008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d4c06d5f0 con 0x7f0d4c15c210 2024-03-29T04:09:28.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.730+0000 7f0d497fa640 1 -- 172.21.15.97:0/1081358008 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7f0d380598c0 con 0x7f0d4c15c210 2024-03-29T04:09:28.710 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:28.710 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2199898540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2199898540},{"type":"v1","addr":"172.21.15.97:6837","nonce":2199898540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1223352413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1223352413},{"type":"v1","addr":"172.21.15.154:6834","nonce":1223352413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/830866132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":830866132},{"type":"v1","addr":"172.21.15.154:6835","nonce":830866132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:08:29.955858+0000","modified":"2024-03-29T04:08:47.764842+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5067":{"gid":5067,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/3142898446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3142898446},{"type":"v1","addr":"172.21.15.97:6835","nonce":3142898446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5067,"qdb_cluster":[5067]},"id":2}]} 2024-03-29T04:09:28.710 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2024-03-29T04:09:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.732+0000 7f0d530c5640 1 -- 172.21.15.97:0/1081358008 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0d3006c560 msgr2=0x7f0d3006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.732+0000 7f0d530c5640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0d3006c560 0x7f0d3006ea20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0d40036000 tx=0x7f0d40030000 comp rx=0 tx=0).stop 2024-03-29T04:09:28.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.732+0000 7f0d530c5640 1 -- 172.21.15.97:0/1081358008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4c15c210 msgr2=0x7f0d4c161170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.732+0000 7f0d530c5640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4c15c210 0x7f0d4c161170 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800e7b0 tx=0x7f0d3800ec70 comp rx=0 tx=0).stop 2024-03-29T04:09:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.732+0000 7f0d530c5640 1 -- 172.21.15.97:0/1081358008 shutdown_connections 2024-03-29T04:09:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.732+0000 7f0d530c5640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0d3006c560 0x7f0d3006ea20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.732+0000 7f0d530c5640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d4c1f4630 0x7f0d4c1f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.732+0000 7f0d530c5640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4c15c210 0x7f0d4c161170 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.732+0000 7f0d530c5640 1 --2- 172.21.15.97:0/1081358008 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d3c0039b0 0x7f0d4c160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.732+0000 7f0d530c5640 1 -- 172.21.15.97:0/1081358008 >> 172.21.15.97:0/1081358008 conn(0x7f0d4c08bfc0 msgr2=0x7f0d4c06db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.732+0000 7f0d530c5640 1 -- 172.21.15.97:0/1081358008 shutdown_connections 2024-03-29T04:09:28.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.732+0000 7f0d530c5640 1 -- 172.21.15.97:0/1081358008 wait complete. 2024-03-29T04:09:28.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:28.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.845+0000 7f10def9d640 1 Processor -- start 2024-03-29T04:09:28.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.845+0000 7f10def9d640 1 -- start start 2024-03-29T04:09:28.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.845+0000 7f10def9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f10d815e930 0x7f10d80cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.845+0000 7f10def9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f10d80d3310 0x7f10d806cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.845+0000 7f10def9d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d80cfb30 con 0x7f10d80cf670 2024-03-29T04:09:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.845+0000 7f10def9d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f10d806c780 con 0x7f10d80d3310 2024-03-29T04:09:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.845+0000 7f10def9d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f10d80cfe10 con 0x7f10d815e930 2024-03-29T04:09:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.845+0000 7f10dd53a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f10d80d3310 0x7f10d806cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.845+0000 7f10dd53a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f10d80d3310 0x7f10d806cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42456/0 (socket says 172.21.15.97:42456) 2024-03-29T04:09:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.846+0000 7f10dd53a640 1 -- 172.21.15.97:0/3863258758 learned_addr learned my addr 172.21.15.97:0/3863258758 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.846+0000 7f10dcd39640 1 --2- 172.21.15.97:0/3863258758 >> v2:172.21.15.154:3300/0 conn(0x7f10d815e930 0x7f10d80cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.846+0000 7f10dd53a640 1 -- 172.21.15.97:0/3863258758 >> v2:172.21.15.154:3300/0 conn(0x7f10d815e930 msgr2=0x7f10d80cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.846+0000 7f10dd53a640 1 --2- 172.21.15.97:0/3863258758 >> v2:172.21.15.154:3300/0 conn(0x7f10d815e930 0x7f10d80cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.846+0000 7f10dd53a640 1 -- 172.21.15.97:0/3863258758 >> v1:172.21.15.154:6789/0 conn(0x7f10d80cf670 legacy=0x7f10d80d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.846+0000 7f10dd53a640 1 -- 172.21.15.97:0/3863258758 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d80d00f0 con 0x7f10d80d3310 2024-03-29T04:09:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.846+0000 7f10dcd39640 1 --2- 172.21.15.97:0/3863258758 >> v2:172.21.15.154:3300/0 conn(0x7f10d815e930 0x7f10d80cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.846+0000 7f10dd53a640 1 --2- 172.21.15.97:0/3863258758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d80d3310 0x7f10d806cc10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f10d808cc70 tx=0x7f10d002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614402d3eb1cb7a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.847+0000 7f10cf7fe640 1 -- 172.21.15.97:0/3863258758 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d002c9c0 con 0x7f10d80d3310 2024-03-29T04:09:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.847+0000 7f10cf7fe640 1 -- 172.21.15.97:0/3863258758 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d002cb60 con 0x7f10d80d3310 2024-03-29T04:09:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.847+0000 7f10cf7fe640 1 -- 172.21.15.97:0/3863258758 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d00343f0 con 0x7f10d80d3310 2024-03-29T04:09:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.847+0000 7f10def9d640 1 -- 172.21.15.97:0/3863258758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d80d3310 msgr2=0x7f10d806cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.847+0000 7f10def9d640 1 --2- 172.21.15.97:0/3863258758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d80d3310 0x7f10d806cc10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f10d808cc70 tx=0x7f10d002be20 comp rx=0 tx=0).stop 2024-03-29T04:09:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.847+0000 7f10def9d640 1 -- 172.21.15.97:0/3863258758 shutdown_connections 2024-03-29T04:09:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.847+0000 7f10def9d640 1 --2- 172.21.15.97:0/3863258758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d80d3310 0x7f10d806cc10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.847+0000 7f10def9d640 1 --2- 172.21.15.97:0/3863258758 >> v2:172.21.15.154:3300/0 conn(0x7f10d815e930 0x7f10d80cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.847+0000 7f10def9d640 1 -- 172.21.15.97:0/3863258758 >> 172.21.15.97:0/3863258758 conn(0x7f10d808bfc0 msgr2=0x7f10d80d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.847+0000 7f10def9d640 1 -- 172.21.15.97:0/3863258758 shutdown_connections 2024-03-29T04:09:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.847+0000 7f10def9d640 1 -- 172.21.15.97:0/3863258758 wait complete. 2024-03-29T04:09:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.849+0000 7f10def9d640 1 Processor -- start 2024-03-29T04:09:28.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.849+0000 7f10def9d640 1 -- start start 2024-03-29T04:09:28.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10def9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d80cf670 0x7f10d81f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10def9d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d80d3310 0x7f10d8160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10def9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10d815e930 0x7f10d8154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10def9d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f10d815ee90 con 0x7f10d80cf670 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10def9d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f10d815ed10 con 0x7f10d815e930 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10def9d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f10d80b3c50 con 0x7f10d80d3310 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10dcd39640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d80cf670 0x7f10d81f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10dcd39640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d80cf670 0x7f10d81f7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42470/0 (socket says 172.21.15.97:42470) 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10dcd39640 1 -- 172.21.15.97:0/536259651 learned_addr learned my addr 172.21.15.97:0/536259651 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10dcd39640 1 -- 172.21.15.97:0/536259651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d80d3310 msgr2=0x7f10d8160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10dcd39640 1 --2- 172.21.15.97:0/536259651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d80d3310 0x7f10d8160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10dcd39640 1 -- 172.21.15.97:0/536259651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10d815e930 msgr2=0x7f10d8154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10dcd39640 1 --2- 172.21.15.97:0/536259651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10d815e930 0x7f10d8154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.850+0000 7f10dcd39640 1 -- 172.21.15.97:0/536259651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d8160bf0 con 0x7f10d80cf670 2024-03-29T04:09:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.851+0000 7f10dcd39640 1 --2- 172.21.15.97:0/536259651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d80cf670 0x7f10d81f7310 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f10d400c7f0 tx=0x7f10d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.851+0000 7f10cd7fa640 1 -- 172.21.15.97:0/536259651 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4015070 con 0x7f10d80cf670 2024-03-29T04:09:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.851+0000 7f10cd7fa640 1 -- 172.21.15.97:0/536259651 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d4007820 con 0x7f10d80cf670 2024-03-29T04:09:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.851+0000 7f10cd7fa640 1 -- 172.21.15.97:0/536259651 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4007bd0 con 0x7f10d80cf670 2024-03-29T04:09:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.851+0000 7f10def9d640 1 -- 172.21.15.97:0/536259651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d8160ed0 con 0x7f10d80cf670 2024-03-29T04:09:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.851+0000 7f10def9d640 1 -- 172.21.15.97:0/536259651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d8161490 con 0x7f10d80cf670 2024-03-29T04:09:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.852+0000 7f10def9d640 1 -- 172.21.15.97:0/536259651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d8068640 con 0x7f10d80cf670 2024-03-29T04:09:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.852+0000 7f10cd7fa640 1 -- 172.21.15.97:0/536259651 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f10d4018070 con 0x7f10d80cf670 2024-03-29T04:09:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.853+0000 7f10cd7fa640 1 --2- 172.21.15.97:0/536259651 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f10c406c530 0x7f10c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.853+0000 7f10cd7fa640 1 -- 172.21.15.97:0/536259651 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f10d408b7a0 con 0x7f10d80cf670 2024-03-29T04:09:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.853+0000 7f10cffff640 1 --2- 172.21.15.97:0/536259651 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f10c406c530 0x7f10c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.853+0000 7f10cffff640 1 --2- 172.21.15.97:0/536259651 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f10c406c530 0x7f10c406e9f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f10d80caa70 tx=0x7f10c80079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:28.857+0000 7f10cd7fa640 1 -- 172.21.15.97:0/536259651 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f10d4059e50 con 0x7f10d80cf670 2024-03-29T04:09:29.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.058+0000 7f10def9d640 1 -- 172.21.15.97:0/536259651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10d80b3130 con 0x7f10d80cf670 2024-03-29T04:09:29.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.059+0000 7f10cd7fa640 1 -- 172.21.15.97:0/536259651 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16669 (secure 0 0 0) 0x7f10d4059750 con 0x7f10d80cf670 2024-03-29T04:09:29.038 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:29.039 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":32,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:08:31.615419+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:08:28.403360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T04:08:28.909709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.061+0000 7f10def9d640 1 -- 172.21.15.97:0/536259651 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f10c406c530 msgr2=0x7f10c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.061+0000 7f10def9d640 1 --2- 172.21.15.97:0/536259651 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f10c406c530 0x7f10c406e9f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f10d80caa70 tx=0x7f10c80079b0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.061+0000 7f10def9d640 1 -- 172.21.15.97:0/536259651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d80cf670 msgr2=0x7f10d81f7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.061+0000 7f10def9d640 1 --2- 172.21.15.97:0/536259651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d80cf670 0x7f10d81f7310 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f10d400c7f0 tx=0x7f10d400ccb0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.062+0000 7f10def9d640 1 -- 172.21.15.97:0/536259651 shutdown_connections 2024-03-29T04:09:29.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.062+0000 7f10def9d640 1 --2- 172.21.15.97:0/536259651 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f10c406c530 0x7f10c406e9f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.062+0000 7f10def9d640 1 --2- 172.21.15.97:0/536259651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10d815e930 0x7f10d8154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.062+0000 7f10def9d640 1 --2- 172.21.15.97:0/536259651 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d80d3310 0x7f10d8160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.062+0000 7f10def9d640 1 --2- 172.21.15.97:0/536259651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10d80cf670 0x7f10d81f7310 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.062+0000 7f10def9d640 1 -- 172.21.15.97:0/536259651 >> 172.21.15.97:0/536259651 conn(0x7f10d808bfc0 msgr2=0x7f10d80b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:29.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.062+0000 7f10def9d640 1 -- 172.21.15.97:0/536259651 shutdown_connections 2024-03-29T04:09:29.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.062+0000 7f10def9d640 1 -- 172.21.15.97:0/536259651 wait complete. 2024-03-29T04:09:29.053 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:07:28.653487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:08:28.403360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:08:28.909709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:09:29.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:29.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.182+0000 7ff0a96d9640 1 Processor -- start 2024-03-29T04:09:29.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.182+0000 7ff0a96d9640 1 -- start start 2024-03-29T04:09:29.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.183+0000 7ff0a96d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a41624b0 0x7ff0a40b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.183+0000 7ff0a96d9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff0a40cca60 0x7ff0a40d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.183+0000 7ff0a96d9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0a406d1b0 con 0x7ff0a40ccea0 2024-03-29T04:09:29.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.183+0000 7ff0a96d9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7ff0a40d0a10 con 0x7ff0a40cca60 2024-03-29T04:09:29.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.183+0000 7ff0a96d9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a406d470 con 0x7ff0a41624b0 2024-03-29T04:09:29.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.184+0000 7ff0a37fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff0a40ccea0 0x7ff0a4162890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51410/0 (socket says 172.21.15.97:51410) 2024-03-29T04:09:29.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.184+0000 7ff0a37fe640 1 -- 172.21.15.97:0/3485984244 learned_addr learned my addr 172.21.15.97:0/3485984244 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:29.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.184+0000 7ff09bfff640 1 --2- 172.21.15.97:0/3485984244 >> v2:172.21.15.154:3300/0 conn(0x7ff0a40cca60 0x7ff0a40d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.184+0000 7ff0a2ffd640 1 --2- 172.21.15.97:0/3485984244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a41624b0 0x7ff0a40b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.184+0000 7ff0a27fc640 1 -- 172.21.15.97:0/3485984244 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3106043885 0 0) 0x7ff0a406d1b0 con 0x7ff0a40ccea0 2024-03-29T04:09:29.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.184+0000 7ff0a27fc640 1 -- 172.21.15.97:0/3485984244 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff090003520 con 0x7ff0a40ccea0 2024-03-29T04:09:29.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.184+0000 7ff09bfff640 1 -- 172.21.15.97:0/3485984244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a41624b0 msgr2=0x7ff0a40b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:29.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.184+0000 7ff09bfff640 1 --2- 172.21.15.97:0/3485984244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a41624b0 0x7ff0a40b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.184+0000 7ff09bfff640 1 -- 172.21.15.97:0/3485984244 >> v1:172.21.15.97:6789/0 conn(0x7ff0a40ccea0 legacy=0x7ff0a4162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.184+0000 7ff0a27fc640 1 -- 172.21.15.97:0/3485984244 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3249109146 0 0) 0x7ff090003520 con 0x7ff0a40ccea0 2024-03-29T04:09:29.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.184+0000 7ff09bfff640 1 -- 172.21.15.97:0/3485984244 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a406d750 con 0x7ff0a40cca60 2024-03-29T04:09:29.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.184+0000 7ff0a2ffd640 1 --2- 172.21.15.97:0/3485984244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a41624b0 0x7ff0a40b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:29.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.185+0000 7ff09bfff640 1 --2- 172.21.15.97:0/3485984244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a40cca60 0x7ff0a40d0ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0097a0 tx=0x7ff08c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e7d5710d3ca3506 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.185+0000 7ff0a27fc640 1 -- 172.21.15.97:0/3485984244 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c037070 con 0x7ff0a40cca60 2024-03-29T04:09:29.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.185+0000 7ff0a27fc640 1 -- 172.21.15.97:0/3485984244 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff08c030510 con 0x7ff0a40cca60 2024-03-29T04:09:29.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.185+0000 7ff0a27fc640 1 -- 172.21.15.97:0/3485984244 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0019f0 con 0x7ff0a40cca60 2024-03-29T04:09:29.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.186+0000 7ff0a96d9640 1 -- 172.21.15.97:0/3485984244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a40cca60 msgr2=0x7ff0a40d0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.186+0000 7ff0a96d9640 1 --2- 172.21.15.97:0/3485984244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a40cca60 0x7ff0a40d0ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0097a0 tx=0x7ff08c02b790 comp rx=0 tx=0).stop 2024-03-29T04:09:29.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.186+0000 7ff0a96d9640 1 -- 172.21.15.97:0/3485984244 shutdown_connections 2024-03-29T04:09:29.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.186+0000 7ff0a96d9640 1 --2- 172.21.15.97:0/3485984244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a40cca60 0x7ff0a40d0ea0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.186+0000 7ff0a96d9640 1 --2- 172.21.15.97:0/3485984244 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a41624b0 0x7ff0a40b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.186+0000 7ff0a96d9640 1 -- 172.21.15.97:0/3485984244 >> 172.21.15.97:0/3485984244 conn(0x7ff0a40b3f00 msgr2=0x7ff0a40b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:29.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.186+0000 7ff0a96d9640 1 -- 172.21.15.97:0/3485984244 shutdown_connections 2024-03-29T04:09:29.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.186+0000 7ff0a96d9640 1 -- 172.21.15.97:0/3485984244 wait complete. 2024-03-29T04:09:29.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.187+0000 7ff0a96d9640 1 Processor -- start 2024-03-29T04:09:29.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.187+0000 7ff0a96d9640 1 -- start start 2024-03-29T04:09:29.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff0a96d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a40cca60 0x7ff0a415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff0a96d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a40ccea0 0x7ff0a40cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff0a96d9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a41624b0 0x7ff0a40c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff0a96d9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a415feb0 con 0x7ff0a40ccea0 2024-03-29T04:09:29.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff0a96d9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a415fd30 con 0x7ff0a41624b0 2024-03-29T04:09:29.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff0a96d9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a4162fa0 con 0x7ff0a40cca60 2024-03-29T04:09:29.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff09bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a40ccea0 0x7ff0a40cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff09bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a40ccea0 0x7ff0a40cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42472/0 (socket says 172.21.15.97:42472) 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff09bfff640 1 -- 172.21.15.97:0/737056515 learned_addr learned my addr 172.21.15.97:0/737056515 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff0a2ffd640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a40cca60 0x7ff0a415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff0a37fe640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a41624b0 0x7ff0a40c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff09bfff640 1 -- 172.21.15.97:0/737056515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a40cca60 msgr2=0x7ff0a415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff09bfff640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a40cca60 0x7ff0a415bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff09bfff640 1 -- 172.21.15.97:0/737056515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a41624b0 msgr2=0x7ff0a40c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff09bfff640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a41624b0 0x7ff0a40c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff09bfff640 1 -- 172.21.15.97:0/737056515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a40ca5e0 con 0x7ff0a40ccea0 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff0a2ffd640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a40cca60 0x7ff0a415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff0a37fe640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a41624b0 0x7ff0a40c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.188+0000 7ff09bfff640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a40ccea0 0x7ff0a40cd560 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff08c02b7d0 tx=0x7ff08c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.189+0000 7ff09b7fe640 1 -- 172.21.15.97:0/737056515 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c041020 con 0x7ff0a40ccea0 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.189+0000 7ff09b7fe640 1 -- 172.21.15.97:0/737056515 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff08c03c870 con 0x7ff0a40ccea0 2024-03-29T04:09:29.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.189+0000 7ff0a96d9640 1 -- 172.21.15.97:0/737056515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a40cbd00 con 0x7ff0a40ccea0 2024-03-29T04:09:29.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.189+0000 7ff09b7fe640 1 -- 172.21.15.97:0/737056515 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c037040 con 0x7ff0a40ccea0 2024-03-29T04:09:29.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.189+0000 7ff0a96d9640 1 -- 172.21.15.97:0/737056515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a40cc2c0 con 0x7ff0a40ccea0 2024-03-29T04:09:29.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.190+0000 7ff0a96d9640 1 -- 172.21.15.97:0/737056515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a40a04c0 con 0x7ff0a40ccea0 2024-03-29T04:09:29.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.190+0000 7ff09b7fe640 1 -- 172.21.15.97:0/737056515 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff08c044020 con 0x7ff0a40ccea0 2024-03-29T04:09:29.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.190+0000 7ff09b7fe640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff08406c530 0x7ff08406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.191+0000 7ff09b7fe640 1 -- 172.21.15.97:0/737056515 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff08c0ad160 con 0x7ff0a40ccea0 2024-03-29T04:09:29.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.191+0000 7ff0a2ffd640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff08406c530 0x7ff08406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.191+0000 7ff0a2ffd640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff08406c530 0x7ff08406e9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff094009a20 tx=0x7ff094008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.198+0000 7ff09b7fe640 1 -- 172.21.15.97:0/737056515 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff08c04b030 con 0x7ff0a40ccea0 2024-03-29T04:09:29.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.380+0000 7ff0a96d9640 1 -- 172.21.15.97:0/737056515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0a40b4a20 con 0x7ff0a40ccea0 2024-03-29T04:09:29.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.381+0000 7ff09b7fe640 1 -- 172.21.15.97:0/737056515 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) ==== 93+0+10 (secure 0 0 0) 0x7ff08c07b770 con 0x7ff0a40ccea0 2024-03-29T04:09:29.360 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-03-29T04:09:29.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.383+0000 7ff0a96d9640 1 -- 172.21.15.97:0/737056515 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff08406c530 msgr2=0x7ff08406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.383+0000 7ff0a96d9640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff08406c530 0x7ff08406e9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff094009a20 tx=0x7ff094008040 comp rx=0 tx=0).stop 2024-03-29T04:09:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.383+0000 7ff0a96d9640 1 -- 172.21.15.97:0/737056515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a40ccea0 msgr2=0x7ff0a40cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.383+0000 7ff0a96d9640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a40ccea0 0x7ff0a40cd560 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff08c02b7d0 tx=0x7ff08c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.383+0000 7ff0a96d9640 1 -- 172.21.15.97:0/737056515 shutdown_connections 2024-03-29T04:09:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.383+0000 7ff0a96d9640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7ff08406c530 0x7ff08406e9f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.383+0000 7ff0a96d9640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0a41624b0 0x7ff0a40c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.383+0000 7ff0a96d9640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a40ccea0 0x7ff0a40cd560 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.383+0000 7ff0a96d9640 1 --2- 172.21.15.97:0/737056515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff0a40cca60 0x7ff0a415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.383+0000 7ff0a96d9640 1 -- 172.21.15.97:0/737056515 >> 172.21.15.97:0/737056515 conn(0x7ff0a40b3f00 msgr2=0x7ff0a4161370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.383+0000 7ff0a96d9640 1 -- 172.21.15.97:0/737056515 shutdown_connections 2024-03-29T04:09:29.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.383+0000 7ff0a96d9640 1 -- 172.21.15.97:0/737056515 wait complete. 2024-03-29T04:09:29.377 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:09:29.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.501+0000 7f0cf9559640 1 Processor -- start 2024-03-29T04:09:29.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.501+0000 7f0cf9559640 1 -- start start 2024-03-29T04:09:29.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf9559640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0cf415e4d0 0x7f0cf406c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf9559640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cf415e8b0 0x7f0cf40d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf9559640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0cf40d0e70 0x7f0cf40d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf9559640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f0cf40cf6b0 con 0x7f0cf415e8b0 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf9559640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f0cf40d13e0 con 0x7f0cf40d0e70 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf9559640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf40d16c0 con 0x7f0cf415e4d0 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf27fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cf415e8b0 0x7f0cf40d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf27fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0cf415e8b0 0x7f0cf40d2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42484/0 (socket says 172.21.15.97:42484) 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf27fc640 1 -- 172.21.15.97:0/129848319 learned_addr learned my addr 172.21.15.97:0/129848319 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf37fe640 1 --2- 172.21.15.97:0/129848319 >> v2:172.21.15.154:3300/0 conn(0x7f0cf40d0e70 0x7f0cf40d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf2ffd640 1 --2- 172.21.15.97:0/129848319 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0cf415e4d0 0x7f0cf406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf27fc640 1 -- 172.21.15.97:0/129848319 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0cf415e4d0 msgr2=0x7f0cf406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf27fc640 1 --2- 172.21.15.97:0/129848319 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0cf415e4d0 0x7f0cf406c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf27fc640 1 -- 172.21.15.97:0/129848319 >> v2:172.21.15.154:3300/0 conn(0x7f0cf40d0e70 msgr2=0x7f0cf40d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf27fc640 1 --2- 172.21.15.97:0/129848319 >> v2:172.21.15.154:3300/0 conn(0x7f0cf40d0e70 0x7f0cf40d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.502+0000 7f0cf27fc640 1 -- 172.21.15.97:0/129848319 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf40d19a0 con 0x7f0cf415e8b0 2024-03-29T04:09:29.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.504+0000 7f0cf27fc640 1 --2- 172.21.15.97:0/129848319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf415e8b0 0x7f0cf40d2350 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40a24b0 tx=0x7f0ce802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fc66791966d756 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.504+0000 7f0cf1ffb640 1 -- 172.21.15.97:0/129848319 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce802d040 con 0x7f0cf415e8b0 2024-03-29T04:09:29.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.504+0000 7f0cf1ffb640 1 -- 172.21.15.97:0/129848319 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce8004230 con 0x7f0cf415e8b0 2024-03-29T04:09:29.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.504+0000 7f0cf1ffb640 1 -- 172.21.15.97:0/129848319 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8003610 con 0x7f0cf415e8b0 2024-03-29T04:09:29.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.504+0000 7f0cf9559640 1 -- 172.21.15.97:0/129848319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf415e8b0 msgr2=0x7f0cf40d2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.504+0000 7f0cf9559640 1 --2- 172.21.15.97:0/129848319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf415e8b0 0x7f0cf40d2350 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40a24b0 tx=0x7f0ce802b8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.504+0000 7f0cf9559640 1 -- 172.21.15.97:0/129848319 shutdown_connections 2024-03-29T04:09:29.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.504+0000 7f0cf9559640 1 --2- 172.21.15.97:0/129848319 >> v2:172.21.15.154:3300/0 conn(0x7f0cf40d0e70 0x7f0cf40d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.504+0000 7f0cf9559640 1 --2- 172.21.15.97:0/129848319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf415e8b0 0x7f0cf40d2350 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.504+0000 7f0cf9559640 1 --2- 172.21.15.97:0/129848319 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0cf415e4d0 0x7f0cf406c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.504+0000 7f0cf9559640 1 -- 172.21.15.97:0/129848319 >> 172.21.15.97:0/129848319 conn(0x7f0cf408bfc0 msgr2=0x7f0cf40d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:29.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.504+0000 7f0cf9559640 1 -- 172.21.15.97:0/129848319 shutdown_connections 2024-03-29T04:09:29.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.504+0000 7f0cf9559640 1 -- 172.21.15.97:0/129848319 wait complete. 2024-03-29T04:09:29.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.506+0000 7f0cf9559640 1 Processor -- start 2024-03-29T04:09:29.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.506+0000 7f0cf9559640 1 -- start start 2024-03-29T04:09:29.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.506+0000 7f0cf9559640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0cf40d0e70 0x7f0cf414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.506+0000 7f0cf9559640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf415e4d0 0x7f0cf40a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.506+0000 7f0cf9559640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0cf415e8b0 0x7f0cf415f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.506+0000 7f0cf9559640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf40d2890 con 0x7f0cf415e4d0 2024-03-29T04:09:29.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.506+0000 7f0cf9559640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf40cf2d0 con 0x7f0cf40d0e70 2024-03-29T04:09:29.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.506+0000 7f0cf9559640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf40a0e60 con 0x7f0cf415e8b0 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.506+0000 7f0cf27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf415e4d0 0x7f0cf40a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.507+0000 7f0cf27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf415e4d0 0x7f0cf40a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42500/0 (socket says 172.21.15.97:42500) 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.507+0000 7f0cf27fc640 1 -- 172.21.15.97:0/2440511756 learned_addr learned my addr 172.21.15.97:0/2440511756 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.507+0000 7f0cf27fc640 1 -- 172.21.15.97:0/2440511756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0cf415e8b0 msgr2=0x7f0cf415f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.507+0000 7f0cf27fc640 1 --2- 172.21.15.97:0/2440511756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0cf415e8b0 0x7f0cf415f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.507+0000 7f0cf27fc640 1 -- 172.21.15.97:0/2440511756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0cf40d0e70 msgr2=0x7f0cf414cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.507+0000 7f0cf27fc640 1 --2- 172.21.15.97:0/2440511756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0cf40d0e70 0x7f0cf414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.507+0000 7f0cf27fc640 1 -- 172.21.15.97:0/2440511756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf40a1140 con 0x7f0cf415e4d0 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.507+0000 7f0cf27fc640 1 --2- 172.21.15.97:0/2440511756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf415e4d0 0x7f0cf40a03a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80090b0 tx=0x7f0ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.507+0000 7f0cdb7fe640 1 -- 172.21.15.97:0/2440511756 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8003c50 con 0x7f0cf415e4d0 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.507+0000 7f0cdb7fe640 1 -- 172.21.15.97:0/2440511756 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce8003df0 con 0x7f0cf415e4d0 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.507+0000 7f0cf9559640 1 -- 172.21.15.97:0/2440511756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf40a1420 con 0x7f0cf415e4d0 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.507+0000 7f0cdb7fe640 1 -- 172.21.15.97:0/2440511756 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8006280 con 0x7f0cf415e4d0 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.507+0000 7f0cf9559640 1 -- 172.21.15.97:0/2440511756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf40a1f00 con 0x7f0cf415e4d0 2024-03-29T04:09:29.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.508+0000 7f0cf9559640 1 -- 172.21.15.97:0/2440511756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf40a2500 con 0x7f0cf415e4d0 2024-03-29T04:09:29.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.509+0000 7f0cdb7fe640 1 -- 172.21.15.97:0/2440511756 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0ce8042020 con 0x7f0cf415e4d0 2024-03-29T04:09:29.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.509+0000 7f0cdb7fe640 1 --2- 172.21.15.97:0/2440511756 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0ce006c530 0x7f0ce006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.509+0000 7f0cdb7fe640 1 -- 172.21.15.97:0/2440511756 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f0ce80ad340 con 0x7f0cf415e4d0 2024-03-29T04:09:29.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.509+0000 7f0cf2ffd640 1 --2- 172.21.15.97:0/2440511756 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0ce006c530 0x7f0ce006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.510+0000 7f0cf2ffd640 1 --2- 172.21.15.97:0/2440511756 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0ce006c530 0x7f0ce006e9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009870 tx=0x7f0ce4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.514+0000 7f0cdb7fe640 1 -- 172.21.15.97:0/2440511756 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0ce807b960 con 0x7f0cf415e4d0 2024-03-29T04:09:29.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.694+0000 7f0cf9559640 1 -- 172.21.15.97:0/2440511756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0cf404f9d0 con 0x7f0cf415e4d0 2024-03-29T04:09:29.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.694+0000 7f0cdb7fe640 1 -- 172.21.15.97:0/2440511756 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) ==== 104+0+11 (secure 0 0 0) 0x7f0ce807b260 con 0x7f0cf415e4d0 2024-03-29T04:09:29.674 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-03-29T04:09:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.696+0000 7f0cf9559640 1 -- 172.21.15.97:0/2440511756 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0ce006c530 msgr2=0x7f0ce006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.696+0000 7f0cf9559640 1 --2- 172.21.15.97:0/2440511756 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0ce006c530 0x7f0ce006e9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009870 tx=0x7f0ce4008040 comp rx=0 tx=0).stop 2024-03-29T04:09:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.696+0000 7f0cf9559640 1 -- 172.21.15.97:0/2440511756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf415e4d0 msgr2=0x7f0cf40a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.696+0000 7f0cf9559640 1 --2- 172.21.15.97:0/2440511756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf415e4d0 0x7f0cf40a03a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80090b0 tx=0x7f0ce80046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.696+0000 7f0cf9559640 1 -- 172.21.15.97:0/2440511756 shutdown_connections 2024-03-29T04:09:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.696+0000 7f0cf9559640 1 --2- 172.21.15.97:0/2440511756 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f0ce006c530 0x7f0ce006e9f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.696+0000 7f0cf9559640 1 --2- 172.21.15.97:0/2440511756 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0cf415e8b0 0x7f0cf415f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.696+0000 7f0cf9559640 1 --2- 172.21.15.97:0/2440511756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cf415e4d0 0x7f0cf40a03a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.696+0000 7f0cf9559640 1 --2- 172.21.15.97:0/2440511756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0cf40d0e70 0x7f0cf414cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.696+0000 7f0cf9559640 1 -- 172.21.15.97:0/2440511756 >> 172.21.15.97:0/2440511756 conn(0x7f0cf408bfc0 msgr2=0x7f0cf4160b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.696+0000 7f0cf9559640 1 -- 172.21.15.97:0/2440511756 shutdown_connections 2024-03-29T04:09:29.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.696+0000 7f0cf9559640 1 -- 172.21.15.97:0/2440511756 wait complete. 2024-03-29T04:09:29.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:09:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.811+0000 7f3a41637640 1 Processor -- start 2024-03-29T04:09:29.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.811+0000 7f3a41637640 1 -- start start 2024-03-29T04:09:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.812+0000 7f3a41637640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3a3c15b4f0 0x7f3a3c0c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.812+0000 7f3a41637640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a3c15bd30 con 0x7f3a3c0b2100 2024-03-29T04:09:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.812+0000 7f3a41637640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a3c15bfa0 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.812+0000 7f3a41637640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f3a3c15c280 con 0x7f3a3c15b4f0 2024-03-29T04:09:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.812+0000 7f3a3b7fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3a3c0b2100 0x7f3a3c0ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51426/0 (socket says 172.21.15.97:51426) 2024-03-29T04:09:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.812+0000 7f3a3b7fe640 1 -- 172.21.15.97:0/2652023065 learned_addr learned my addr 172.21.15.97:0/2652023065 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.812+0000 7f3a3affd640 1 --2- 172.21.15.97:0/2652023065 >> v2:172.21.15.154:3300/0 conn(0x7f3a3c15b4f0 0x7f3a3c0c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.812+0000 7f3a3a7fc640 1 -- 172.21.15.97:0/2652023065 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1351249503 0 0) 0x7f3a3c15bd30 con 0x7f3a3c0b2100 2024-03-29T04:09:29.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.812+0000 7f3a3a7fc640 1 -- 172.21.15.97:0/2652023065 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a28003520 con 0x7f3a3c0b2100 2024-03-29T04:09:29.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.812+0000 7f3a3affd640 1 -- 172.21.15.97:0/2652023065 >> v1:172.21.15.154:6789/0 conn(0x7f3a3c15b8d0 legacy=0x7f3a3c0ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.812+0000 7f3a3affd640 1 -- 172.21.15.97:0/2652023065 >> v1:172.21.15.97:6789/0 conn(0x7f3a3c0b2100 legacy=0x7f3a3c0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.812+0000 7f3a3affd640 1 -- 172.21.15.97:0/2652023065 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3c15c560 con 0x7f3a3c15b4f0 2024-03-29T04:09:29.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.813+0000 7f3a3affd640 1 --2- 172.21.15.97:0/2652023065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a3c15b4f0 0x7f3a3c0c6c50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0097a0 tx=0x7f3a2c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8df1bc9bbb98347 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.813+0000 7f3a3a7fc640 1 -- 172.21.15.97:0/2652023065 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c035020 con 0x7f3a3c15b4f0 2024-03-29T04:09:29.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.813+0000 7f3a3a7fc640 1 -- 172.21.15.97:0/2652023065 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a2c031830 con 0x7f3a3c15b4f0 2024-03-29T04:09:29.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.813+0000 7f3a3a7fc640 1 -- 172.21.15.97:0/2652023065 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c031be0 con 0x7f3a3c15b4f0 2024-03-29T04:09:29.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.814+0000 7f3a41637640 1 -- 172.21.15.97:0/2652023065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a3c15b4f0 msgr2=0x7f3a3c0c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.814+0000 7f3a41637640 1 --2- 172.21.15.97:0/2652023065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a3c15b4f0 0x7f3a3c0c6c50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0097a0 tx=0x7f3a2c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.814+0000 7f3a41637640 1 -- 172.21.15.97:0/2652023065 shutdown_connections 2024-03-29T04:09:29.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.814+0000 7f3a41637640 1 --2- 172.21.15.97:0/2652023065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a3c15b4f0 0x7f3a3c0c6c50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.814+0000 7f3a41637640 1 -- 172.21.15.97:0/2652023065 >> 172.21.15.97:0/2652023065 conn(0x7f3a3c0d0450 msgr2=0x7f3a3c0d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:29.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.814+0000 7f3a41637640 1 -- 172.21.15.97:0/2652023065 shutdown_connections 2024-03-29T04:09:29.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.814+0000 7f3a41637640 1 -- 172.21.15.97:0/2652023065 wait complete. 2024-03-29T04:09:29.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.815+0000 7f3a41637640 1 Processor -- start 2024-03-29T04:09:29.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.815+0000 7f3a41637640 1 -- start start 2024-03-29T04:09:29.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a41637640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a3c0b2100 0x7f3a3c0c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a41637640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a3c15b4f0 0x7f3a3c1477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a41637640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a3c15b8d0 0x7f3a3c073100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a41637640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c0b4d50 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a41637640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c0b4bd0 con 0x7f3a3c0b2100 2024-03-29T04:09:29.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a41637640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3a3c0ce880 con 0x7f3a3c15b4f0 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a3b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a3c15b8d0 0x7f3a3c073100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a3b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a3c15b8d0 0x7f3a3c073100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42504/0 (socket says 172.21.15.97:42504) 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a3b7fe640 1 -- 172.21.15.97:0/448281402 learned_addr learned my addr 172.21.15.97:0/448281402 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a3affd640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a3c0b2100 0x7f3a3c0c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a33fff640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a3c15b4f0 0x7f3a3c1477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a3b7fe640 1 -- 172.21.15.97:0/448281402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a3c15b4f0 msgr2=0x7f3a3c1477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a3b7fe640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a3c15b4f0 0x7f3a3c1477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a3b7fe640 1 -- 172.21.15.97:0/448281402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a3c0b2100 msgr2=0x7f3a3c0c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a3b7fe640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a3c0b2100 0x7f3a3c0c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a3b7fe640 1 -- 172.21.15.97:0/448281402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3c158a10 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a3b7fe640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a3c15b8d0 0x7f3a3c073100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400cb70 tx=0x7f3a3400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a337fe640 1 -- 172.21.15.97:0/448281402 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34017070 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a41637640 1 -- 172.21.15.97:0/448281402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3c158cf0 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a337fe640 1 -- 172.21.15.97:0/448281402 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a3400ad40 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a337fe640 1 -- 172.21.15.97:0/448281402 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34012620 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.816+0000 7f3a41637640 1 -- 172.21.15.97:0/448281402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a3c1592b0 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.817+0000 7f3a41637640 1 -- 172.21.15.97:0/448281402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a3c09e520 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.818+0000 7f3a337fe640 1 -- 172.21.15.97:0/448281402 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3a3401a070 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.818+0000 7f3a337fe640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3a1c06c530 0x7f3a1c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.818+0000 7f3a337fe640 1 -- 172.21.15.97:0/448281402 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3a3408b400 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.818+0000 7f3a3affd640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3a1c06c530 0x7f3a1c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.819+0000 7f3a3affd640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3a1c06c530 0x7f3a1c06e9f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c036000 tx=0x7f3a2c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:29.827+0000 7f3a337fe640 1 -- 172.21.15.97:0/448281402 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3a340593e0 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.006+0000 7f3a41637640 1 -- 172.21.15.97:0/448281402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3a3c04f9d0 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.006+0000 7f3a337fe640 1 -- 172.21.15.97:0/448281402 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) ==== 100+0+11 (secure 0 0 0) 0x7f3a3405d6e0 con 0x7f3a3c15b8d0 2024-03-29T04:09:29.986 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-03-29T04:09:29.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.008+0000 7f3a41637640 1 -- 172.21.15.97:0/448281402 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3a1c06c530 msgr2=0x7f3a1c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.008+0000 7f3a41637640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3a1c06c530 0x7f3a1c06e9f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c036000 tx=0x7f3a2c008030 comp rx=0 tx=0).stop 2024-03-29T04:09:29.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.008+0000 7f3a41637640 1 -- 172.21.15.97:0/448281402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a3c15b8d0 msgr2=0x7f3a3c073100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.008+0000 7f3a41637640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a3c15b8d0 0x7f3a3c073100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400cb70 tx=0x7f3a3400a430 comp rx=0 tx=0).stop 2024-03-29T04:09:29.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.008+0000 7f3a41637640 1 -- 172.21.15.97:0/448281402 shutdown_connections 2024-03-29T04:09:29.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.008+0000 7f3a41637640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3a1c06c530 0x7f3a1c06e9f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.008+0000 7f3a41637640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a3c15b8d0 0x7f3a3c073100 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.008+0000 7f3a41637640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3a3c15b4f0 0x7f3a3c1477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.008+0000 7f3a41637640 1 --2- 172.21.15.97:0/448281402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a3c0b2100 0x7f3a3c0c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.008+0000 7f3a41637640 1 -- 172.21.15.97:0/448281402 >> 172.21.15.97:0/448281402 conn(0x7f3a3c0d0450 msgr2=0x7f3a3c0ad1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:29.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.008+0000 7f3a41637640 1 -- 172.21.15.97:0/448281402 shutdown_connections 2024-03-29T04:09:29.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.008+0000 7f3a41637640 1 -- 172.21.15.97:0/448281402 wait complete. 2024-03-29T04:09:30.002 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:30.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.124+0000 7f2ab3435640 1 Processor -- start 2024-03-29T04:09:30.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.124+0000 7f2ab3435640 1 -- start start 2024-03-29T04:09:30.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab3435640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2aac15b370 0x7f2aac0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab3435640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2aac073290 0x7f2aac15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab3435640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aac071ae0 con 0x7f2aac06f3f0 2024-03-29T04:09:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab3435640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f2aac071dc0 con 0x7f2aac073290 2024-03-29T04:09:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab3435640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f2aac0720a0 con 0x7f2aac15b370 2024-03-29T04:09:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab19d2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2aac073290 0x7f2aac15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab19d2640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2aac073290 0x7f2aac15f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42508/0 (socket says 172.21.15.97:42508) 2024-03-29T04:09:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab09d0640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2aac06f3f0 0x7f2aac072b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51434/0 (socket says 172.21.15.97:51434) 2024-03-29T04:09:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab19d2640 1 -- 172.21.15.97:0/1340589902 learned_addr learned my addr 172.21.15.97:0/1340589902 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab11d1640 1 --2- 172.21.15.97:0/1340589902 >> v2:172.21.15.154:3300/0 conn(0x7f2aac15b370 0x7f2aac0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab19d2640 1 -- 172.21.15.97:0/1340589902 >> v2:172.21.15.154:3300/0 conn(0x7f2aac15b370 msgr2=0x7f2aac0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab19d2640 1 --2- 172.21.15.97:0/1340589902 >> v2:172.21.15.154:3300/0 conn(0x7f2aac15b370 0x7f2aac0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab19d2640 1 -- 172.21.15.97:0/1340589902 >> v1:172.21.15.97:6789/0 conn(0x7f2aac06f3f0 legacy=0x7f2aac072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2ab19d2640 1 -- 172.21.15.97:0/1340589902 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aac072380 con 0x7f2aac073290 2024-03-29T04:09:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.126+0000 7f2aa3fff640 1 -- 172.21.15.97:0/1340589902 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2956701609 0 0) 0x7f2aac071ae0 con 0x7f2aac06f3f0 2024-03-29T04:09:30.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.127+0000 7f2ab19d2640 1 --2- 172.21.15.97:0/1340589902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aac073290 0x7f2aac15f0a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0d12e0 tx=0x7f2aa402e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ac988ca3f69fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.127+0000 7f2aa3fff640 1 -- 172.21.15.97:0/1340589902 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400b0d0 con 0x7f2aac073290 2024-03-29T04:09:30.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.127+0000 7f2aa3fff640 1 -- 172.21.15.97:0/1340589902 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa400b270 con 0x7f2aac073290 2024-03-29T04:09:30.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.127+0000 7f2aa3fff640 1 -- 172.21.15.97:0/1340589902 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4009600 con 0x7f2aac073290 2024-03-29T04:09:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.127+0000 7f2ab3435640 1 -- 172.21.15.97:0/1340589902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aac073290 msgr2=0x7f2aac15f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.127+0000 7f2ab3435640 1 --2- 172.21.15.97:0/1340589902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aac073290 0x7f2aac15f0a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0d12e0 tx=0x7f2aa402e180 comp rx=0 tx=0).stop 2024-03-29T04:09:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.128+0000 7f2ab3435640 1 -- 172.21.15.97:0/1340589902 shutdown_connections 2024-03-29T04:09:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.128+0000 7f2ab3435640 1 --2- 172.21.15.97:0/1340589902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aac073290 0x7f2aac15f0a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.128+0000 7f2ab3435640 1 --2- 172.21.15.97:0/1340589902 >> v2:172.21.15.154:3300/0 conn(0x7f2aac15b370 0x7f2aac0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.128+0000 7f2ab3435640 1 -- 172.21.15.97:0/1340589902 >> 172.21.15.97:0/1340589902 conn(0x7f2aac06b510 msgr2=0x7f2aac06b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.128+0000 7f2ab3435640 1 -- 172.21.15.97:0/1340589902 shutdown_connections 2024-03-29T04:09:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.128+0000 7f2ab3435640 1 -- 172.21.15.97:0/1340589902 wait complete. 2024-03-29T04:09:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.129+0000 7f2ab3435640 1 Processor -- start 2024-03-29T04:09:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.129+0000 7f2ab3435640 1 -- start start 2024-03-29T04:09:30.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.129+0000 7f2ab3435640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aac06f3f0 0x7f2aac0cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.129+0000 7f2ab3435640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2aac073290 0x7f2aac14a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.129+0000 7f2ab3435640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aac15b370 0x7f2aac0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.129+0000 7f2ab3435640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac0b3980 con 0x7f2aac15b370 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab3435640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac0b3800 con 0x7f2aac073290 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab3435640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2aac15b750 con 0x7f2aac06f3f0 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab19d2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aac15b370 0x7f2aac0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab19d2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aac15b370 0x7f2aac0ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42522/0 (socket says 172.21.15.97:42522) 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab19d2640 1 -- 172.21.15.97:0/3217012513 learned_addr learned my addr 172.21.15.97:0/3217012513 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab11d1640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aac06f3f0 0x7f2aac0cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab09d0640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2aac073290 0x7f2aac14a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab19d2640 1 -- 172.21.15.97:0/3217012513 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aac06f3f0 msgr2=0x7f2aac0cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab19d2640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aac06f3f0 0x7f2aac0cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab19d2640 1 -- 172.21.15.97:0/3217012513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2aac073290 msgr2=0x7f2aac14a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab19d2640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2aac073290 0x7f2aac14a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab19d2640 1 -- 172.21.15.97:0/3217012513 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aac0cd1e0 con 0x7f2aac15b370 2024-03-29T04:09:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab19d2640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aac15b370 0x7f2aac0ccca0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4038040 tx=0x7f2aa4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2aa1ffb640 1 -- 172.21.15.97:0/3217012513 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400b0d0 con 0x7f2aac15b370 2024-03-29T04:09:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2aa1ffb640 1 -- 172.21.15.97:0/3217012513 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa4045070 con 0x7f2aac15b370 2024-03-29T04:09:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab3435640 1 -- 172.21.15.97:0/3217012513 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aac0cc490 con 0x7f2aac15b370 2024-03-29T04:09:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2aa1ffb640 1 -- 172.21.15.97:0/3217012513 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400b720 con 0x7f2aac15b370 2024-03-29T04:09:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.130+0000 7f2ab3435640 1 -- 172.21.15.97:0/3217012513 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aac156a70 con 0x7f2aac15b370 2024-03-29T04:09:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.131+0000 7f2ab3435640 1 -- 172.21.15.97:0/3217012513 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aac068640 con 0x7f2aac15b370 2024-03-29T04:09:30.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.132+0000 7f2aa1ffb640 1 -- 172.21.15.97:0/3217012513 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2aa4036bf0 con 0x7f2aac15b370 2024-03-29T04:09:30.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.132+0000 7f2aa1ffb640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2a9006c530 0x7f2a9006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.133+0000 7f2aa1ffb640 1 -- 172.21.15.97:0/3217012513 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2aa40ae5e0 con 0x7f2aac15b370 2024-03-29T04:09:30.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.133+0000 7f2ab11d1640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2a9006c530 0x7f2a9006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.133+0000 7f2ab11d1640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2a9006c530 0x7f2a9006e9f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80079a0 tx=0x7f2aa8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.137+0000 7f2aa1ffb640 1 -- 172.21.15.97:0/3217012513 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2aa407c760 con 0x7f2aac15b370 2024-03-29T04:09:30.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.348+0000 7f2ab3435640 1 -- 172.21.15.97:0/3217012513 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2aac158720 con 0x7f2aac15b370 2024-03-29T04:09:30.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.348+0000 7f2aa1ffb640 1 -- 172.21.15.97:0/3217012513 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7f2aa4080a60 con 0x7f2aac15b370 2024-03-29T04:09:30.328 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:30.328 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2199898540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2199898540},{"type":"v1","addr":"172.21.15.97:6837","nonce":2199898540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1223352413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1223352413},{"type":"v1","addr":"172.21.15.154:6834","nonce":1223352413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/830866132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":830866132},{"type":"v1","addr":"172.21.15.154:6835","nonce":830866132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:08:29.955858+0000","modified":"2024-03-29T04:08:47.764842+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5067":{"gid":5067,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/3142898446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3142898446},{"type":"v1","addr":"172.21.15.97:6835","nonce":3142898446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5067,"qdb_cluster":[5067]},"id":2}]} 2024-03-29T04:09:30.329 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2024-03-29T04:09:30.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.350+0000 7f2ab3435640 1 -- 172.21.15.97:0/3217012513 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2a9006c530 msgr2=0x7f2a9006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.350+0000 7f2ab3435640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2a9006c530 0x7f2a9006e9f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80079a0 tx=0x7f2aa8008040 comp rx=0 tx=0).stop 2024-03-29T04:09:30.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.350+0000 7f2ab3435640 1 -- 172.21.15.97:0/3217012513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aac15b370 msgr2=0x7f2aac0ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.350+0000 7f2ab3435640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aac15b370 0x7f2aac0ccca0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4038040 tx=0x7f2aa4004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:30.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.351+0000 7f2ab3435640 1 -- 172.21.15.97:0/3217012513 shutdown_connections 2024-03-29T04:09:30.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.351+0000 7f2ab3435640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f2a9006c530 0x7f2a9006e9f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.351+0000 7f2ab3435640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aac15b370 0x7f2aac0ccca0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.351+0000 7f2ab3435640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2aac073290 0x7f2aac14a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.351+0000 7f2ab3435640 1 --2- 172.21.15.97:0/3217012513 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2aac06f3f0 0x7f2aac0cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.351+0000 7f2ab3435640 1 -- 172.21.15.97:0/3217012513 >> 172.21.15.97:0/3217012513 conn(0x7f2aac06b510 msgr2=0x7f2aac06dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:30.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.351+0000 7f2ab3435640 1 -- 172.21.15.97:0/3217012513 shutdown_connections 2024-03-29T04:09:30.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.351+0000 7f2ab3435640 1 -- 172.21.15.97:0/3217012513 wait complete. 2024-03-29T04:09:30.342 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.464+0000 7f48e9e63640 1 Processor -- start 2024-03-29T04:09:30.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.464+0000 7f48e9e63640 1 -- start start 2024-03-29T04:09:30.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e9e63640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48e415e930 0x7f48e40cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e9e63640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48e40cf670 0x7f48e40d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e9e63640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48e40cfb30 con 0x7f48e40b3c50 2024-03-29T04:09:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e9e63640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f48e406c780 con 0x7f48e415e930 2024-03-29T04:09:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e9e63640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f48e40cfe10 con 0x7f48e40cf670 2024-03-29T04:09:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e37fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48e415e930 0x7f48e40cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e3fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f48e40b3c50 0x7f48e40d0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:51438/0 (socket says 172.21.15.97:51438) 2024-03-29T04:09:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e3fff640 1 -- 172.21.15.97:0/4050242002 learned_addr learned my addr 172.21.15.97:0/4050242002 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e37fe640 1 --2- 172.21.15.97:0/4050242002 >> v2:172.21.15.97:3300/0 conn(0x7f48e415e930 0x7f48e40cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42532/0 (socket says 172.21.15.97:42532) 2024-03-29T04:09:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e2ffd640 1 --2- 172.21.15.97:0/4050242002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48e40cf670 0x7f48e40d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e37fe640 1 -- 172.21.15.97:0/4050242002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48e40cf670 msgr2=0x7f48e40d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e37fe640 1 --2- 172.21.15.97:0/4050242002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48e40cf670 0x7f48e40d2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e37fe640 1 -- 172.21.15.97:0/4050242002 >> v1:172.21.15.97:6789/0 conn(0x7f48e40b3c50 legacy=0x7f48e40d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e37fe640 1 -- 172.21.15.97:0/4050242002 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e40d00f0 con 0x7f48e415e930 2024-03-29T04:09:30.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.465+0000 7f48e27fc640 1 -- 172.21.15.97:0/4050242002 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2269730690 0 0) 0x7f48e40cfb30 con 0x7f48e40b3c50 2024-03-29T04:09:30.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.466+0000 7f48e37fe640 1 --2- 172.21.15.97:0/4050242002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e415e930 0x7f48e40cf130 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f48d80097a0 tx=0x7f48d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c5fc089333d9e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.466+0000 7f48e27fc640 1 -- 172.21.15.97:0/4050242002 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d80315a0 con 0x7f48e415e930 2024-03-29T04:09:30.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.466+0000 7f48e27fc640 1 -- 172.21.15.97:0/4050242002 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48d8037070 con 0x7f48e415e930 2024-03-29T04:09:30.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.466+0000 7f48e27fc640 1 -- 172.21.15.97:0/4050242002 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8031960 con 0x7f48e415e930 2024-03-29T04:09:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.467+0000 7f48e9e63640 1 -- 172.21.15.97:0/4050242002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e415e930 msgr2=0x7f48e40cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.467+0000 7f48e9e63640 1 --2- 172.21.15.97:0/4050242002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e415e930 0x7f48e40cf130 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f48d80097a0 tx=0x7f48d802b840 comp rx=0 tx=0).stop 2024-03-29T04:09:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.467+0000 7f48e9e63640 1 -- 172.21.15.97:0/4050242002 shutdown_connections 2024-03-29T04:09:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.467+0000 7f48e9e63640 1 --2- 172.21.15.97:0/4050242002 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48e40cf670 0x7f48e40d2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.467+0000 7f48e9e63640 1 --2- 172.21.15.97:0/4050242002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e415e930 0x7f48e40cf130 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.467+0000 7f48e9e63640 1 -- 172.21.15.97:0/4050242002 >> 172.21.15.97:0/4050242002 conn(0x7f48e408bfc0 msgr2=0x7f48e40d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.467+0000 7f48e9e63640 1 -- 172.21.15.97:0/4050242002 shutdown_connections 2024-03-29T04:09:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.467+0000 7f48e9e63640 1 -- 172.21.15.97:0/4050242002 wait complete. 2024-03-29T04:09:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.468+0000 7f48e9e63640 1 Processor -- start 2024-03-29T04:09:30.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.468+0000 7f48e9e63640 1 -- start start 2024-03-29T04:09:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e9e63640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48e40b3c50 0x7f48e40a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e9e63640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48e40cf670 0x7f48e414fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e9e63640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e415e930 0x7f48e409f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e9e63640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f48e415ee90 con 0x7f48e415e930 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e9e63640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f48e415ed10 con 0x7f48e40cf670 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e9e63640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f48e406dc20 con 0x7f48e40b3c50 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e415e930 0x7f48e409f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e415e930 0x7f48e409f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42548/0 (socket says 172.21.15.97:42548) 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e3fff640 1 -- 172.21.15.97:0/1239973957 learned_addr learned my addr 172.21.15.97:0/1239973957 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e2ffd640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48e40cf670 0x7f48e414fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e37fe640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48e40b3c50 0x7f48e40a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e3fff640 1 -- 172.21.15.97:0/1239973957 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48e40b3c50 msgr2=0x7f48e40a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e3fff640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48e40b3c50 0x7f48e40a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e3fff640 1 -- 172.21.15.97:0/1239973957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48e40cf670 msgr2=0x7f48e414fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e3fff640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48e40cf670 0x7f48e414fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.469+0000 7f48e3fff640 1 -- 172.21.15.97:0/1239973957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e41f4d90 con 0x7f48e415e930 2024-03-29T04:09:30.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.470+0000 7f48e3fff640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e415e930 0x7f48e409f0e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00c730 tx=0x7f48dc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.470+0000 7f48cbfff640 1 -- 172.21.15.97:0/1239973957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc018070 con 0x7f48e415e930 2024-03-29T04:09:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.470+0000 7f48cbfff640 1 -- 172.21.15.97:0/1239973957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48dc00ce40 con 0x7f48e415e930 2024-03-29T04:09:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.470+0000 7f48e9e63640 1 -- 172.21.15.97:0/1239973957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e409e600 con 0x7f48e415e930 2024-03-29T04:09:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.470+0000 7f48cbfff640 1 -- 172.21.15.97:0/1239973957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc00a860 con 0x7f48e415e930 2024-03-29T04:09:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.470+0000 7f48e9e63640 1 -- 172.21.15.97:0/1239973957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e414ff40 con 0x7f48e415e930 2024-03-29T04:09:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.471+0000 7f48e9e63640 1 -- 172.21.15.97:0/1239973957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e409e3f0 con 0x7f48e415e930 2024-03-29T04:09:30.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.471+0000 7f48cbfff640 1 -- 172.21.15.97:0/1239973957 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f48dc01b070 con 0x7f48e415e930 2024-03-29T04:09:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.472+0000 7f48cbfff640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f48d006c530 0x7f48d006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.472+0000 7f48e37fe640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f48d006c530 0x7f48d006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.472+0000 7f48cbfff640 1 -- 172.21.15.97:0/1239973957 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f48dc08ac00 con 0x7f48e415e930 2024-03-29T04:09:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.472+0000 7f48e37fe640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f48d006c530 0x7f48d006e9f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f48d8036000 tx=0x7f48d803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.480+0000 7f48cbfff640 1 -- 172.21.15.97:0/1239973957 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f48dc091080 con 0x7f48e415e930 2024-03-29T04:09:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.664+0000 7f48e9e63640 1 -- 172.21.15.97:0/1239973957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48e40cb0a0 con 0x7f48e415e930 2024-03-29T04:09:30.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.665+0000 7f48cbfff640 1 -- 172.21.15.97:0/1239973957 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16669 (secure 0 0 0) 0x7f48dc058bd0 con 0x7f48e415e930 2024-03-29T04:09:30.646 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:30.647 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":32,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:08:31.615419+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:08:28.403360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T04:08:28.909709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:30.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.667+0000 7f48e9e63640 1 -- 172.21.15.97:0/1239973957 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f48d006c530 msgr2=0x7f48d006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.667+0000 7f48e9e63640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f48d006c530 0x7f48d006e9f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f48d8036000 tx=0x7f48d803d040 comp rx=0 tx=0).stop 2024-03-29T04:09:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.667+0000 7f48e9e63640 1 -- 172.21.15.97:0/1239973957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e415e930 msgr2=0x7f48e409f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.667+0000 7f48e9e63640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e415e930 0x7f48e409f0e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00c730 tx=0x7f48dc00cbf0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.667+0000 7f48e9e63640 1 -- 172.21.15.97:0/1239973957 shutdown_connections 2024-03-29T04:09:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.667+0000 7f48e9e63640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f48d006c530 0x7f48d006e9f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.667+0000 7f48e9e63640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e415e930 0x7f48e409f0e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.667+0000 7f48e9e63640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48e40cf670 0x7f48e414fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.667+0000 7f48e9e63640 1 --2- 172.21.15.97:0/1239973957 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48e40b3c50 0x7f48e40a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.667+0000 7f48e9e63640 1 -- 172.21.15.97:0/1239973957 >> 172.21.15.97:0/1239973957 conn(0x7f48e408bfc0 msgr2=0x7f48e40d49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.667+0000 7f48e9e63640 1 -- 172.21.15.97:0/1239973957 shutdown_connections 2024-03-29T04:09:30.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.667+0000 7f48e9e63640 1 -- 172.21.15.97:0/1239973957 wait complete. 2024-03-29T04:09:30.658 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T04:09:30.659 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.818+0000 7f23a2c7e640 1 Processor -- start 2024-03-29T04:09:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.818+0000 7f23a2c7e640 1 -- start start 2024-03-29T04:09:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.819+0000 7f23a2c7e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f239c0b3c50 0x7f239c0d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.819+0000 7f23a2c7e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f239c0d3140 0x7f239c06cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.819+0000 7f23a2c7e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f239c0cfb30 con 0x7f239c15e930 2024-03-29T04:09:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.819+0000 7f23a2c7e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f239c06c780 con 0x7f239c0b3c50 2024-03-29T04:09:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.819+0000 7f23a2c7e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f239c0cfe10 con 0x7f239c0d3140 2024-03-29T04:09:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.820+0000 7f23a121b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f239c0d3140 0x7f239c06cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.820+0000 7f239bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f239c0b3c50 0x7f239c0d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.820+0000 7f23a121b640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f239c0d3140 0x7f239c06cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:37018/0 (socket says 172.21.15.97:37018) 2024-03-29T04:09:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.820+0000 7f239bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f239c0b3c50 0x7f239c0d2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:52838/0 (socket says 172.21.15.97:52838) 2024-03-29T04:09:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.820+0000 7f23a121b640 1 -- 172.21.15.97:0/3745472462 learned_addr learned my addr 172.21.15.97:0/3745472462 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:30.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.820+0000 7f23a121b640 1 -- 172.21.15.97:0/3745472462 >> v2:172.21.15.154:3300/0 conn(0x7f239c0b3c50 msgr2=0x7f239c0d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:30.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.820+0000 7f23a121b640 1 --2- 172.21.15.97:0/3745472462 >> v2:172.21.15.154:3300/0 conn(0x7f239c0b3c50 0x7f239c0d2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.820+0000 7f23a121b640 1 -- 172.21.15.97:0/3745472462 >> v1:172.21.15.154:6789/0 conn(0x7f239c15e930 legacy=0x7f239c0cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.820+0000 7f23a121b640 1 -- 172.21.15.97:0/3745472462 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c0d00f0 con 0x7f239c0d3140 2024-03-29T04:09:30.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.821+0000 7f23a121b640 1 --2- 172.21.15.97:0/3745472462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f239c0d3140 0x7f239c06cc10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2394009e50 tx=0x7f239402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c81331ce280c2d43 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.821+0000 7f239b7fe640 1 -- 172.21.15.97:0/3745472462 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239402ca70 con 0x7f239c0d3140 2024-03-29T04:09:30.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.821+0000 7f239b7fe640 1 -- 172.21.15.97:0/3745472462 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239402cc10 con 0x7f239c0d3140 2024-03-29T04:09:30.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.821+0000 7f239b7fe640 1 -- 172.21.15.97:0/3745472462 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23940343f0 con 0x7f239c0d3140 2024-03-29T04:09:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.821+0000 7f23a2c7e640 1 -- 172.21.15.97:0/3745472462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f239c0d3140 msgr2=0x7f239c06cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.821+0000 7f23a2c7e640 1 --2- 172.21.15.97:0/3745472462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f239c0d3140 0x7f239c06cc10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2394009e50 tx=0x7f239402bed0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.821+0000 7f23a2c7e640 1 -- 172.21.15.97:0/3745472462 shutdown_connections 2024-03-29T04:09:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.821+0000 7f23a2c7e640 1 --2- 172.21.15.97:0/3745472462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f239c0d3140 0x7f239c06cc10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.821+0000 7f23a2c7e640 1 --2- 172.21.15.97:0/3745472462 >> v2:172.21.15.154:3300/0 conn(0x7f239c0b3c50 0x7f239c0d2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.821+0000 7f23a2c7e640 1 -- 172.21.15.97:0/3745472462 >> 172.21.15.97:0/3745472462 conn(0x7f239c08bfc0 msgr2=0x7f239c0d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.822+0000 7f23a2c7e640 1 -- 172.21.15.97:0/3745472462 shutdown_connections 2024-03-29T04:09:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.822+0000 7f23a2c7e640 1 -- 172.21.15.97:0/3745472462 wait complete. 2024-03-29T04:09:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.823+0000 7f23a2c7e640 1 Processor -- start 2024-03-29T04:09:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.823+0000 7f23a2c7e640 1 -- start start 2024-03-29T04:09:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.823+0000 7f23a2c7e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f239c0b3c50 0x7f239c0ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a2c7e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239c0d3140 0x7f239c157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a2c7e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f239c15e930 0x7f239c157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a2c7e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f239c15ed10 con 0x7f239c15e930 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a2c7e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f239c0cf840 con 0x7f239c0d3140 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a2c7e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f239c15ee90 con 0x7f239c0b3c50 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a121b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f239c15e930 0x7f239c157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a121b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f239c15e930 0x7f239c157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36052/0 (socket says 172.21.15.97:36052) 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a121b640 1 -- 172.21.15.97:0/189095628 learned_addr learned my addr 172.21.15.97:0/189095628 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f239bfff640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239c0d3140 0x7f239c157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a0a1a640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f239c0b3c50 0x7f239c0ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a121b640 1 -- 172.21.15.97:0/189095628 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f239c0b3c50 msgr2=0x7f239c0ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a121b640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f239c0b3c50 0x7f239c0ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a121b640 1 -- 172.21.15.97:0/189095628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239c0d3140 msgr2=0x7f239c157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a121b640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239c0d3140 0x7f239c157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a121b640 1 -- 172.21.15.97:0/189095628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c0cad30 con 0x7f239c15e930 2024-03-29T04:09:30.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a121b640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f239c15e930 0x7f239c157fc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2394036040 tx=0x7f2394004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23997fa640 1 -- 172.21.15.97:0/189095628 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394043020 con 0x7f239c15e930 2024-03-29T04:09:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23997fa640 1 -- 172.21.15.97:0/189095628 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239403e450 con 0x7f239c15e930 2024-03-29T04:09:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a2c7e640 1 -- 172.21.15.97:0/189095628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239c15a6b0 con 0x7f239c15e930 2024-03-29T04:09:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23997fa640 1 -- 172.21.15.97:0/189095628 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394009510 con 0x7f239c15e930 2024-03-29T04:09:30.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.824+0000 7f23a2c7e640 1 -- 172.21.15.97:0/189095628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239c15ac70 con 0x7f239c15e930 2024-03-29T04:09:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.825+0000 7f23a2c7e640 1 -- 172.21.15.97:0/189095628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f239c068640 con 0x7f239c15e930 2024-03-29T04:09:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.826+0000 7f23997fa640 1 -- 172.21.15.97:0/189095628 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2394046020 con 0x7f239c15e930 2024-03-29T04:09:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.826+0000 7f23997fa640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f238006c530 0x7f238006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.826+0000 7f23a0a1a640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f238006c530 0x7f238006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.826+0000 7f23997fa640 1 -- 172.21.15.97:0/189095628 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f23940adcd0 con 0x7f239c15e930 2024-03-29T04:09:30.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.827+0000 7f23a0a1a640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f238006c530 0x7f238006e9f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2390001780 tx=0x7f2390009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:30.831+0000 7f23997fa640 1 -- 172.21.15.97:0/189095628 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f239404d030 con 0x7f239c15e930 2024-03-29T04:09:31.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.050+0000 7f23a2c7e640 1 -- 172.21.15.97:0/189095628 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f239c0b37f0 con 0x7f239c15e930 2024-03-29T04:09:31.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.050+0000 7f23997fa640 1 -- 172.21.15.97:0/189095628 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7f239407bcb0 con 0x7f239c15e930 2024-03-29T04:09:31.030 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:31.030 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2199898540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2199898540},{"type":"v1","addr":"172.21.15.97:6837","nonce":2199898540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1223352413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1223352413},{"type":"v1","addr":"172.21.15.154:6834","nonce":1223352413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/830866132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":830866132},{"type":"v1","addr":"172.21.15.154:6835","nonce":830866132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:08:29.955858+0000","modified":"2024-03-29T04:08:47.764842+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5067":{"gid":5067,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/3142898446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3142898446},{"type":"v1","addr":"172.21.15.97:6835","nonce":3142898446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5067,"qdb_cluster":[5067]},"id":2}]} 2024-03-29T04:09:31.031 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2024-03-29T04:09:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.052+0000 7f23a2c7e640 1 -- 172.21.15.97:0/189095628 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f238006c530 msgr2=0x7f238006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.052+0000 7f23a2c7e640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f238006c530 0x7f238006e9f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2390001780 tx=0x7f2390009040 comp rx=0 tx=0).stop 2024-03-29T04:09:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.052+0000 7f23a2c7e640 1 -- 172.21.15.97:0/189095628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f239c15e930 msgr2=0x7f239c157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.052+0000 7f23a2c7e640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f239c15e930 0x7f239c157fc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2394036040 tx=0x7f2394004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.052+0000 7f23a2c7e640 1 -- 172.21.15.97:0/189095628 shutdown_connections 2024-03-29T04:09:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.052+0000 7f23a2c7e640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f238006c530 0x7f238006e9f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.052+0000 7f23a2c7e640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f239c15e930 0x7f239c157fc0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.052+0000 7f23a2c7e640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239c0d3140 0x7f239c157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.052+0000 7f23a2c7e640 1 --2- 172.21.15.97:0/189095628 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f239c0b3c50 0x7f239c0ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.052+0000 7f23a2c7e640 1 -- 172.21.15.97:0/189095628 >> 172.21.15.97:0/189095628 conn(0x7f239c08bfc0 msgr2=0x7f239c15dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.053+0000 7f23a2c7e640 1 -- 172.21.15.97:0/189095628 shutdown_connections 2024-03-29T04:09:31.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.053+0000 7f23a2c7e640 1 -- 172.21.15.97:0/189095628 wait complete. 2024-03-29T04:09:31.043 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:31.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.164+0000 7f66bf1f0640 1 Processor -- start 2024-03-29T04:09:31.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.164+0000 7f66bf1f0640 1 -- start start 2024-03-29T04:09:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.165+0000 7f66bf1f0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66b809b6e0 0x7f66b81623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.165+0000 7f66bf1f0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b80d1a10 con 0x7f66b815e6e0 2024-03-29T04:09:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.165+0000 7f66bf1f0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b80d0e90 con 0x7f66b81628e0 2024-03-29T04:09:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.165+0000 7f66bf1f0640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f66b809bb40 con 0x7f66b809b6e0 2024-03-29T04:09:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.165+0000 7f66bcf8c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f66b815e6e0 0x7f66b80d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57370/0 (socket says 172.21.15.97:57370) 2024-03-29T04:09:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.165+0000 7f66bcf8c640 1 -- 172.21.15.97:0/1733953791 learned_addr learned my addr 172.21.15.97:0/1733953791 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:31.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.165+0000 7f66b7fff640 1 --2- 172.21.15.97:0/1733953791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66b809b6e0 0x7f66b81623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.165+0000 7f66b77fe640 1 -- 172.21.15.97:0/1733953791 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1371475492 0 0) 0x7f66b80d1a10 con 0x7f66b815e6e0 2024-03-29T04:09:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.165+0000 7f66b77fe640 1 -- 172.21.15.97:0/1733953791 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66a4003520 con 0x7f66b815e6e0 2024-03-29T04:09:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.166+0000 7f66b77fe640 1 -- 172.21.15.97:0/1733953791 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2467515031 0 0) 0x7f66b80d0e90 con 0x7f66b81628e0 2024-03-29T04:09:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.166+0000 7f66b7fff640 1 -- 172.21.15.97:0/1733953791 >> v1:172.21.15.154:6789/0 conn(0x7f66b81628e0 legacy=0x7f66b80b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.166+0000 7f66b7fff640 1 -- 172.21.15.97:0/1733953791 >> v1:172.21.15.97:6789/0 conn(0x7f66b815e6e0 legacy=0x7f66b80d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.166+0000 7f66b7fff640 1 -- 172.21.15.97:0/1733953791 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b80d1170 con 0x7f66b809b6e0 2024-03-29T04:09:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.166+0000 7f66b7fff640 1 --2- 172.21.15.97:0/1733953791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66b809b6e0 0x7f66b81623a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009800 tx=0x7f66a002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0a6b4831e929d61 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.167+0000 7f66b77fe640 1 -- 172.21.15.97:0/1733953791 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0035020 con 0x7f66b809b6e0 2024-03-29T04:09:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.167+0000 7f66b77fe640 1 -- 172.21.15.97:0/1733953791 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a0031890 con 0x7f66b809b6e0 2024-03-29T04:09:31.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.167+0000 7f66b77fe640 1 -- 172.21.15.97:0/1733953791 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0031c60 con 0x7f66b809b6e0 2024-03-29T04:09:31.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.167+0000 7f66bf1f0640 1 -- 172.21.15.97:0/1733953791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66b809b6e0 msgr2=0x7f66b81623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.167+0000 7f66bf1f0640 1 --2- 172.21.15.97:0/1733953791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66b809b6e0 0x7f66b81623a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009800 tx=0x7f66a002b930 comp rx=0 tx=0).stop 2024-03-29T04:09:31.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.167+0000 7f66bf1f0640 1 -- 172.21.15.97:0/1733953791 shutdown_connections 2024-03-29T04:09:31.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.167+0000 7f66bf1f0640 1 --2- 172.21.15.97:0/1733953791 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66b809b6e0 0x7f66b81623a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.167+0000 7f66bf1f0640 1 -- 172.21.15.97:0/1733953791 >> 172.21.15.97:0/1733953791 conn(0x7f66b80b3f10 msgr2=0x7f66b80b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:31.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.167+0000 7f66bf1f0640 1 -- 172.21.15.97:0/1733953791 shutdown_connections 2024-03-29T04:09:31.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.167+0000 7f66bf1f0640 1 -- 172.21.15.97:0/1733953791 wait complete. 2024-03-29T04:09:31.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.168+0000 7f66bf1f0640 1 Processor -- start 2024-03-29T04:09:31.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.168+0000 7f66bf1f0640 1 -- start start 2024-03-29T04:09:31.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.169+0000 7f66bf1f0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66b809b6e0 0x7f66b81f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.169+0000 7f66bf1f0640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66b815e6e0 0x7f66b80b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.169+0000 7f66bf1f0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b81628e0 0x7f66b80b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.169+0000 7f66bf1f0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66b80b1d80 con 0x7f66b81628e0 2024-03-29T04:09:31.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.169+0000 7f66bf1f0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f66b80d1730 con 0x7f66b809b6e0 2024-03-29T04:09:31.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.169+0000 7f66bf1f0640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f66b806d910 con 0x7f66b815e6e0 2024-03-29T04:09:31.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.169+0000 7f66bd78d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b81628e0 0x7f66b80b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.169+0000 7f66bd78d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b81628e0 0x7f66b80b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36060/0 (socket says 172.21.15.97:36060) 2024-03-29T04:09:31.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.169+0000 7f66bd78d640 1 -- 172.21.15.97:0/2233209668 learned_addr learned my addr 172.21.15.97:0/2233209668 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:31.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66b7fff640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66b815e6e0 0x7f66b80b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66bcf8c640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66b809b6e0 0x7f66b81f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66bd78d640 1 -- 172.21.15.97:0/2233209668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66b815e6e0 msgr2=0x7f66b80b7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66bd78d640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66b815e6e0 0x7f66b80b7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66bd78d640 1 -- 172.21.15.97:0/2233209668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66b809b6e0 msgr2=0x7f66b81f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66bd78d640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66b809b6e0 0x7f66b81f7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66bd78d640 1 -- 172.21.15.97:0/2233209668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b815f470 con 0x7f66b81628e0 2024-03-29T04:09:31.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66b7fff640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66b815e6e0 0x7f66b80b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:31.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66bd78d640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b81628e0 0x7f66b80b6dd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c760 tx=0x7f66b000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:31.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66b57fa640 1 -- 172.21.15.97:0/2233209668 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0003da0 con 0x7f66b81628e0 2024-03-29T04:09:31.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66bf1f0640 1 -- 172.21.15.97:0/2233209668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b815f750 con 0x7f66b81628e0 2024-03-29T04:09:31.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66b57fa640 1 -- 172.21.15.97:0/2233209668 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b0005070 con 0x7f66b81628e0 2024-03-29T04:09:31.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66b57fa640 1 -- 172.21.15.97:0/2233209668 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0005460 con 0x7f66b81628e0 2024-03-29T04:09:31.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.170+0000 7f66bf1f0640 1 -- 172.21.15.97:0/2233209668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b815fd10 con 0x7f66b81628e0 2024-03-29T04:09:31.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.171+0000 7f66bf1f0640 1 -- 172.21.15.97:0/2233209668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b8068640 con 0x7f66b81628e0 2024-03-29T04:09:31.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.171+0000 7f66b57fa640 1 -- 172.21.15.97:0/2233209668 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f66b0005600 con 0x7f66b81628e0 2024-03-29T04:09:31.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.172+0000 7f66b57fa640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f669806c530 0x7f669806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.172+0000 7f66bcf8c640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f669806c530 0x7f669806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.172+0000 7f66b57fa640 1 -- 172.21.15.97:0/2233209668 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f66b008b6b0 con 0x7f66b81628e0 2024-03-29T04:09:31.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.172+0000 7f66bcf8c640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f669806c530 0x7f669806e9f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f66b815ee80 tx=0x7f66a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:31.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.176+0000 7f66b57fa640 1 -- 172.21.15.97:0/2233209668 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f66b0059c50 con 0x7f66b81628e0 2024-03-29T04:09:31.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.389+0000 7f66bf1f0640 1 -- 172.21.15.97:0/2233209668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66b80b4b00 con 0x7f66b81628e0 2024-03-29T04:09:31.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.389+0000 7f66b57fa640 1 -- 172.21.15.97:0/2233209668 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7f66b0059550 con 0x7f66b81628e0 2024-03-29T04:09:31.369 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:31.369 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2199898540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2199898540},{"type":"v1","addr":"172.21.15.97:6837","nonce":2199898540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1223352413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1223352413},{"type":"v1","addr":"172.21.15.154:6834","nonce":1223352413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/830866132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":830866132},{"type":"v1","addr":"172.21.15.154:6835","nonce":830866132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:08:29.955858+0000","modified":"2024-03-29T04:08:47.764842+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5067":{"gid":5067,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/3142898446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3142898446},{"type":"v1","addr":"172.21.15.97:6835","nonce":3142898446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5067,"qdb_cluster":[5067]},"id":2}]} 2024-03-29T04:09:31.369 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 16 2024-03-29T04:09:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.391+0000 7f66bf1f0640 1 -- 172.21.15.97:0/2233209668 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f669806c530 msgr2=0x7f669806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.391+0000 7f66bf1f0640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f669806c530 0x7f669806e9f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f66b815ee80 tx=0x7f66a8009040 comp rx=0 tx=0).stop 2024-03-29T04:09:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.391+0000 7f66bf1f0640 1 -- 172.21.15.97:0/2233209668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b81628e0 msgr2=0x7f66b80b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.391+0000 7f66bf1f0640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b81628e0 0x7f66b80b6dd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c760 tx=0x7f66b000cc20 comp rx=0 tx=0).stop 2024-03-29T04:09:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.391+0000 7f66bf1f0640 1 -- 172.21.15.97:0/2233209668 shutdown_connections 2024-03-29T04:09:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.391+0000 7f66bf1f0640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f669806c530 0x7f669806e9f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.391+0000 7f66bf1f0640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b81628e0 0x7f66b80b6dd0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.391+0000 7f66bf1f0640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66b815e6e0 0x7f66b80b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.391+0000 7f66bf1f0640 1 --2- 172.21.15.97:0/2233209668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66b809b6e0 0x7f66b81f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.391+0000 7f66bf1f0640 1 -- 172.21.15.97:0/2233209668 >> 172.21.15.97:0/2233209668 conn(0x7f66b80b3f10 msgr2=0x7f66b80b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:31.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.391+0000 7f66bf1f0640 1 -- 172.21.15.97:0/2233209668 shutdown_connections 2024-03-29T04:09:31.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.391+0000 7f66bf1f0640 1 -- 172.21.15.97:0/2233209668 wait complete. 2024-03-29T04:09:31.381 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:31.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.503+0000 7fec602c1640 1 Processor -- start 2024-03-29T04:09:31.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.503+0000 7fec602c1640 1 -- start start 2024-03-29T04:09:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.504+0000 7fec602c1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fec580ae1b0 0x7fec5806b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.504+0000 7fec602c1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec580ae630 0x7fec5806ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.504+0000 7fec602c1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec580b41d0 con 0x7fec5815e870 2024-03-29T04:09:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.504+0000 7fec602c1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fec580b6510 con 0x7fec580ae1b0 2024-03-29T04:09:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.504+0000 7fec602c1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fec580b4470 con 0x7fec580ae630 2024-03-29T04:09:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.505+0000 7fec5d85c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fec580ae1b0 0x7fec5806b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.505+0000 7fec5d85c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fec580ae1b0 0x7fec5806b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36068/0 (socket says 172.21.15.97:36068) 2024-03-29T04:09:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.505+0000 7fec5d85c640 1 -- 172.21.15.97:0/928840159 learned_addr learned my addr 172.21.15.97:0/928840159 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.505+0000 7fec5e85e640 1 --2- 172.21.15.97:0/928840159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec580ae630 0x7fec5806ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.505+0000 7fec5d85c640 1 -- 172.21.15.97:0/928840159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec580ae630 msgr2=0x7fec5806ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.505+0000 7fec5d85c640 1 --2- 172.21.15.97:0/928840159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec580ae630 0x7fec5806ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.505+0000 7fec5d85c640 1 -- 172.21.15.97:0/928840159 >> v1:172.21.15.154:6789/0 conn(0x7fec5815e870 legacy=0x7fec580b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.505+0000 7fec5d85c640 1 -- 172.21.15.97:0/928840159 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec580b4750 con 0x7fec580ae1b0 2024-03-29T04:09:31.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.506+0000 7fec5d85c640 1 --2- 172.21.15.97:0/928840159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec580ae1b0 0x7fec5806b510 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fec48009870 tx=0x7fec4802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aede7a1ce4cee3f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:31.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.506+0000 7fec5d05b640 1 -- 172.21.15.97:0/928840159 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48035020 con 0x7fec580ae1b0 2024-03-29T04:09:31.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.506+0000 7fec5d05b640 1 -- 172.21.15.97:0/928840159 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec48031640 con 0x7fec580ae1b0 2024-03-29T04:09:31.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.506+0000 7fec5d05b640 1 -- 172.21.15.97:0/928840159 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4802d040 con 0x7fec580ae1b0 2024-03-29T04:09:31.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.506+0000 7fec602c1640 1 -- 172.21.15.97:0/928840159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec580ae1b0 msgr2=0x7fec5806b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.506+0000 7fec602c1640 1 --2- 172.21.15.97:0/928840159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec580ae1b0 0x7fec5806b510 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fec48009870 tx=0x7fec4802b8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.506+0000 7fec602c1640 1 -- 172.21.15.97:0/928840159 shutdown_connections 2024-03-29T04:09:31.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.506+0000 7fec602c1640 1 --2- 172.21.15.97:0/928840159 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec580ae630 0x7fec5806ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.506+0000 7fec602c1640 1 --2- 172.21.15.97:0/928840159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec580ae1b0 0x7fec5806b510 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.506+0000 7fec602c1640 1 -- 172.21.15.97:0/928840159 >> 172.21.15.97:0/928840159 conn(0x7fec580b66f0 msgr2=0x7fec580b6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:31.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.506+0000 7fec602c1640 1 -- 172.21.15.97:0/928840159 shutdown_connections 2024-03-29T04:09:31.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.506+0000 7fec602c1640 1 -- 172.21.15.97:0/928840159 wait complete. 2024-03-29T04:09:31.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.508+0000 7fec602c1640 1 Processor -- start 2024-03-29T04:09:31.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.508+0000 7fec602c1640 1 -- start start 2024-03-29T04:09:31.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.509+0000 7fec602c1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec580ae1b0 0x7fec58142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.509+0000 7fec602c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec580ae630 0x7fec58159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.509+0000 7fec602c1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec5815e870 0x7fec58145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.509+0000 7fec602c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fec580b04c0 con 0x7fec580ae630 2024-03-29T04:09:31.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.509+0000 7fec602c1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fec5815ec50 con 0x7fec5815e870 2024-03-29T04:09:31.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.509+0000 7fec602c1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fec580b5d50 con 0x7fec580ae1b0 2024-03-29T04:09:31.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec5d85c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec580ae630 0x7fec58159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec5d85c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec580ae630 0x7fec58159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36082/0 (socket says 172.21.15.97:36082) 2024-03-29T04:09:31.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec5d85c640 1 -- 172.21.15.97:0/161595914 learned_addr learned my addr 172.21.15.97:0/161595914 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:31.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec5e05d640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec580ae1b0 0x7fec58142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec5e85e640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec5815e870 0x7fec58145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec5d85c640 1 -- 172.21.15.97:0/161595914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec580ae1b0 msgr2=0x7fec58142990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec5d85c640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec580ae1b0 0x7fec58142990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec5d85c640 1 -- 172.21.15.97:0/161595914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec5815e870 msgr2=0x7fec58145340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec5d85c640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec5815e870 0x7fec58145340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec5d85c640 1 -- 172.21.15.97:0/161595914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec58143380 con 0x7fec580ae630 2024-03-29T04:09:31.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec5e85e640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec5815e870 0x7fec58145340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:31.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec5e05d640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec580ae1b0 0x7fec58142990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:31.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec5d85c640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec580ae630 0x7fec58159c90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fec48000c00 tx=0x7fec480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:31.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec4effd640 1 -- 172.21.15.97:0/161595914 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48040070 con 0x7fec580ae630 2024-03-29T04:09:31.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec4effd640 1 -- 172.21.15.97:0/161595914 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec480399f0 con 0x7fec580ae630 2024-03-29T04:09:31.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec602c1640 1 -- 172.21.15.97:0/161595914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5815a670 con 0x7fec580ae630 2024-03-29T04:09:31.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec4effd640 1 -- 172.21.15.97:0/161595914 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48035040 con 0x7fec580ae630 2024-03-29T04:09:31.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.510+0000 7fec602c1640 1 -- 172.21.15.97:0/161595914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec58143620 con 0x7fec580ae630 2024-03-29T04:09:31.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.511+0000 7fec602c1640 1 -- 172.21.15.97:0/161595914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec58068d50 con 0x7fec580ae630 2024-03-29T04:09:31.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.512+0000 7fec4effd640 1 -- 172.21.15.97:0/161595914 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fec4804b060 con 0x7fec580ae630 2024-03-29T04:09:31.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.512+0000 7fec4effd640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fec3c06c530 0x7fec3c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.512+0000 7fec4effd640 1 -- 172.21.15.97:0/161595914 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fec480acf30 con 0x7fec580ae630 2024-03-29T04:09:31.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.512+0000 7fec5e05d640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fec3c06c530 0x7fec3c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.513+0000 7fec5e05d640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fec3c06c530 0x7fec3c06e9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fec54000f60 tx=0x7fec54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:31.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.517+0000 7fec4effd640 1 -- 172.21.15.97:0/161595914 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fec4807b550 con 0x7fec580ae630 2024-03-29T04:09:31.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.740+0000 7fec602c1640 1 -- 172.21.15.97:0/161595914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec580b3d30 con 0x7fec580ae630 2024-03-29T04:09:31.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.741+0000 7fec4effd640 1 -- 172.21.15.97:0/161595914 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16669 (secure 0 0 0) 0x7fec4807ae50 con 0x7fec580ae630 2024-03-29T04:09:31.721 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:31.721 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":32,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:08:31.615419+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:08:28.403360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T04:08:28.909709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.743+0000 7fec602c1640 1 -- 172.21.15.97:0/161595914 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fec3c06c530 msgr2=0x7fec3c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.743+0000 7fec602c1640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fec3c06c530 0x7fec3c06e9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fec54000f60 tx=0x7fec54009040 comp rx=0 tx=0).stop 2024-03-29T04:09:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.743+0000 7fec602c1640 1 -- 172.21.15.97:0/161595914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec580ae630 msgr2=0x7fec58159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.743+0000 7fec602c1640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec580ae630 0x7fec58159c90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fec48000c00 tx=0x7fec480046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.743+0000 7fec602c1640 1 -- 172.21.15.97:0/161595914 shutdown_connections 2024-03-29T04:09:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.743+0000 7fec602c1640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fec3c06c530 0x7fec3c06e9f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.743+0000 7fec602c1640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec5815e870 0x7fec58145340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.743+0000 7fec602c1640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec580ae630 0x7fec58159c90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.743+0000 7fec602c1640 1 --2- 172.21.15.97:0/161595914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec580ae1b0 0x7fec58142990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.743+0000 7fec602c1640 1 -- 172.21.15.97:0/161595914 >> 172.21.15.97:0/161595914 conn(0x7fec580b66f0 msgr2=0x7fec580aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.743+0000 7fec602c1640 1 -- 172.21.15.97:0/161595914 shutdown_connections 2024-03-29T04:09:31.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.743+0000 7fec602c1640 1 -- 172.21.15.97:0/161595914 wait complete. 2024-03-29T04:09:31.734 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T04:09:31.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.857+0000 7fd66163f640 1 Processor -- start 2024-03-29T04:09:31.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.857+0000 7fd66163f640 1 -- start start 2024-03-29T04:09:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd66163f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd65c15eb60 0x7fd65c0cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd66163f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd65c0ba000 0x7fd65c0d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd66163f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd65c0cfe70 con 0x7fd65c15e780 2024-03-29T04:09:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd66163f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fd65c0b7f50 con 0x7fd65c15eb60 2024-03-29T04:09:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd66163f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd65c0b8230 con 0x7fd65c0ba000 2024-03-29T04:09:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd653fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd65c15eb60 0x7fd65c0cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd65affd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd65c15e780 0x7fd65c0cebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57380/0 (socket says 172.21.15.97:57380) 2024-03-29T04:09:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd653fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd65c15eb60 0x7fd65c0cf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36084/0 (socket says 172.21.15.97:36084) 2024-03-29T04:09:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd65affd640 1 -- 172.21.15.97:0/2363974292 learned_addr learned my addr 172.21.15.97:0/2363974292 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd65b7fe640 1 --2- 172.21.15.97:0/2363974292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd65c0ba000 0x7fd65c0d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd653fff640 1 -- 172.21.15.97:0/2363974292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd65c0ba000 msgr2=0x7fd65c0d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:31.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd653fff640 1 --2- 172.21.15.97:0/2363974292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd65c0ba000 0x7fd65c0d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd653fff640 1 -- 172.21.15.97:0/2363974292 >> v1:172.21.15.97:6789/0 conn(0x7fd65c15e780 legacy=0x7fd65c0cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd653fff640 1 -- 172.21.15.97:0/2363974292 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65c0b8510 con 0x7fd65c15eb60 2024-03-29T04:09:31.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.858+0000 7fd65a7fc640 1 -- 172.21.15.97:0/2363974292 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2865695588 0 0) 0x7fd65c0cfe70 con 0x7fd65c15e780 2024-03-29T04:09:31.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.859+0000 7fd653fff640 1 --2- 172.21.15.97:0/2363974292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd65c15eb60 0x7fd65c0cf2d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd6440097a0 tx=0x7fd64402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fbb20248db1e89 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:31.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.859+0000 7fd65a7fc640 1 -- 172.21.15.97:0/2363974292 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd644035020 con 0x7fd65c15eb60 2024-03-29T04:09:31.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.859+0000 7fd65a7fc640 1 -- 172.21.15.97:0/2363974292 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd644031830 con 0x7fd65c15eb60 2024-03-29T04:09:31.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.859+0000 7fd65a7fc640 1 -- 172.21.15.97:0/2363974292 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd644031c00 con 0x7fd65c15eb60 2024-03-29T04:09:31.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.860+0000 7fd66163f640 1 -- 172.21.15.97:0/2363974292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd65c15eb60 msgr2=0x7fd65c0cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.860+0000 7fd66163f640 1 --2- 172.21.15.97:0/2363974292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd65c15eb60 0x7fd65c0cf2d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd6440097a0 tx=0x7fd64402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.860+0000 7fd66163f640 1 -- 172.21.15.97:0/2363974292 shutdown_connections 2024-03-29T04:09:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.860+0000 7fd66163f640 1 --2- 172.21.15.97:0/2363974292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd65c0ba000 0x7fd65c0d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.860+0000 7fd66163f640 1 --2- 172.21.15.97:0/2363974292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd65c15eb60 0x7fd65c0cf2d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.860+0000 7fd66163f640 1 -- 172.21.15.97:0/2363974292 >> 172.21.15.97:0/2363974292 conn(0x7fd65c08bfc0 msgr2=0x7fd65c0d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.860+0000 7fd66163f640 1 -- 172.21.15.97:0/2363974292 shutdown_connections 2024-03-29T04:09:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.860+0000 7fd66163f640 1 -- 172.21.15.97:0/2363974292 wait complete. 2024-03-29T04:09:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.861+0000 7fd66163f640 1 Processor -- start 2024-03-29T04:09:31.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.861+0000 7fd66163f640 1 -- start start 2024-03-29T04:09:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd66163f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd65c0ba000 0x7fd65c09e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd66163f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd65c15e780 0x7fd65c0a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd66163f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd65c1f7650 0x7fd65c1f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd66163f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c0cf810 con 0x7fd65c15e780 2024-03-29T04:09:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd66163f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c0b8ad0 con 0x7fd65c1f7650 2024-03-29T04:09:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd66163f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd65c0cf990 con 0x7fd65c0ba000 2024-03-29T04:09:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd653fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd65c15e780 0x7fd65c0a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd653fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd65c15e780 0x7fd65c0a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36098/0 (socket says 172.21.15.97:36098) 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd653fff640 1 -- 172.21.15.97:0/2648611264 learned_addr learned my addr 172.21.15.97:0/2648611264 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd65affd640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd65c0ba000 0x7fd65c09e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd65b7fe640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd65c1f7650 0x7fd65c1f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd653fff640 1 -- 172.21.15.97:0/2648611264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd65c0ba000 msgr2=0x7fd65c09e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd653fff640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd65c0ba000 0x7fd65c09e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd653fff640 1 -- 172.21.15.97:0/2648611264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd65c1f7650 msgr2=0x7fd65c1f7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd653fff640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd65c1f7650 0x7fd65c1f7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd653fff640 1 -- 172.21.15.97:0/2648611264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65c09e920 con 0x7fd65c15e780 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd653fff640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd65c15e780 0x7fd65c0a0fa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd6440097a0 tx=0x7fd6440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd6537fe640 1 -- 172.21.15.97:0/2648611264 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd644035040 con 0x7fd65c15e780 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd6537fe640 1 -- 172.21.15.97:0/2648611264 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd644004230 con 0x7fd65c15e780 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd6537fe640 1 -- 172.21.15.97:0/2648611264 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64403a700 con 0x7fd65c15e780 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd66163f640 1 -- 172.21.15.97:0/2648611264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65c09ec00 con 0x7fd65c15e780 2024-03-29T04:09:31.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.862+0000 7fd66163f640 1 -- 172.21.15.97:0/2648611264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65c09f1c0 con 0x7fd65c15e780 2024-03-29T04:09:31.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.864+0000 7fd6537fe640 1 -- 172.21.15.97:0/2648611264 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd644038020 con 0x7fd65c15e780 2024-03-29T04:09:31.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.864+0000 7fd66163f640 1 -- 172.21.15.97:0/2648611264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd65c068640 con 0x7fd65c15e780 2024-03-29T04:09:31.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.864+0000 7fd6537fe640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd63c06c530 0x7fd63c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.864+0000 7fd65affd640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd63c06c530 0x7fd63c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.864+0000 7fd6537fe640 1 -- 172.21.15.97:0/2648611264 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd6440ad480 con 0x7fd65c15e780 2024-03-29T04:09:31.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.865+0000 7fd65affd640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd63c06c530 0x7fd63c06e9f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd64c000f60 tx=0x7fd64c007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:31.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:31.869+0000 7fd6537fe640 1 -- 172.21.15.97:0/2648611264 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd64407ba00 con 0x7fd65c15e780 2024-03-29T04:09:32.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.087+0000 7fd66163f640 1 -- 172.21.15.97:0/2648611264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd65c0d2730 con 0x7fd65c15e780 2024-03-29T04:09:32.067 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-03-29T04:09:32.087+0000 7f7c3447e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:09:32.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.089+0000 7fd6537fe640 1 -- 172.21.15.97:0/2648611264 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7fd64407b300 con 0x7fd65c15e780 2024-03-29T04:09:32.068 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T04:09:32.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.092+0000 7fd66163f640 1 -- 172.21.15.97:0/2648611264 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd63c06c530 msgr2=0x7fd63c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.092+0000 7fd66163f640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd63c06c530 0x7fd63c06e9f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd64c000f60 tx=0x7fd64c007d50 comp rx=0 tx=0).stop 2024-03-29T04:09:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.092+0000 7fd66163f640 1 -- 172.21.15.97:0/2648611264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd65c15e780 msgr2=0x7fd65c0a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.092+0000 7fd66163f640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd65c15e780 0x7fd65c0a0fa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd6440097a0 tx=0x7fd6440046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.092+0000 7fd66163f640 1 -- 172.21.15.97:0/2648611264 shutdown_connections 2024-03-29T04:09:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.092+0000 7fd66163f640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd63c06c530 0x7fd63c06e9f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.092+0000 7fd66163f640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd65c1f7650 0x7fd65c1f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.092+0000 7fd66163f640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd65c15e780 0x7fd65c0a0fa0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.092+0000 7fd66163f640 1 --2- 172.21.15.97:0/2648611264 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd65c0ba000 0x7fd65c09e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.092+0000 7fd66163f640 1 -- 172.21.15.97:0/2648611264 >> 172.21.15.97:0/2648611264 conn(0x7fd65c08bfc0 msgr2=0x7fd65c15dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.092+0000 7fd66163f640 1 -- 172.21.15.97:0/2648611264 shutdown_connections 2024-03-29T04:09:32.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.092+0000 7fd66163f640 1 -- 172.21.15.97:0/2648611264 wait complete. 2024-03-29T04:09:32.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T04:09:32.107 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-03-29T04:09:32.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.206+0000 7f67421c8640 1 Processor -- start 2024-03-29T04:09:32.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.206+0000 7f67421c8640 1 -- start start 2024-03-29T04:09:32.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.207+0000 7f67421c8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f673c09b930 0x7f673c09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.207+0000 7f67421c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f673c09af70 0x7f673c15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.207+0000 7f67421c8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673c0d0e10 con 0x7f673c070990 2024-03-29T04:09:32.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.207+0000 7f67421c8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f673c09bdc0 con 0x7f673c09b930 2024-03-29T04:09:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.207+0000 7f67421c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f673c0d10f0 con 0x7f673c09af70 2024-03-29T04:09:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.207+0000 7f673affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f673c09b930 0x7f673c09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.207+0000 7f673affd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f673c09b930 0x7f673c09aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36124/0 (socket says 172.21.15.97:36124) 2024-03-29T04:09:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.207+0000 7f673affd640 1 -- 172.21.15.97:0/1763511330 learned_addr learned my addr 172.21.15.97:0/1763511330 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.207+0000 7f673bfff640 1 --2- 172.21.15.97:0/1763511330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f673c09af70 0x7f673c15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.208+0000 7f673affd640 1 -- 172.21.15.97:0/1763511330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f673c09af70 msgr2=0x7f673c15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.208+0000 7f673affd640 1 --2- 172.21.15.97:0/1763511330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f673c09af70 0x7f673c15e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.208+0000 7f673affd640 1 -- 172.21.15.97:0/1763511330 >> v1:172.21.15.154:6789/0 conn(0x7f673c070990 legacy=0x7f673c0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.208+0000 7f673affd640 1 -- 172.21.15.97:0/1763511330 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c0d13d0 con 0x7f673c09b930 2024-03-29T04:09:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.208+0000 7f673affd640 1 --2- 172.21.15.97:0/1763511330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c09b930 0x7f673c09aa30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6730009870 tx=0x7f673002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458c59e8ec95864a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.208+0000 7f673a7fc640 1 -- 172.21.15.97:0/1763511330 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730034070 con 0x7f673c09b930 2024-03-29T04:09:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.208+0000 7f673a7fc640 1 -- 172.21.15.97:0/1763511330 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6730030840 con 0x7f673c09b930 2024-03-29T04:09:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.208+0000 7f673a7fc640 1 -- 172.21.15.97:0/1763511330 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730030c10 con 0x7f673c09b930 2024-03-29T04:09:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.209+0000 7f67421c8640 1 -- 172.21.15.97:0/1763511330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c09b930 msgr2=0x7f673c09aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.209+0000 7f67421c8640 1 --2- 172.21.15.97:0/1763511330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c09b930 0x7f673c09aa30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6730009870 tx=0x7f673002b980 comp rx=0 tx=0).stop 2024-03-29T04:09:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.209+0000 7f67421c8640 1 -- 172.21.15.97:0/1763511330 shutdown_connections 2024-03-29T04:09:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.209+0000 7f67421c8640 1 --2- 172.21.15.97:0/1763511330 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f673c09af70 0x7f673c15e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.209+0000 7f67421c8640 1 --2- 172.21.15.97:0/1763511330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c09b930 0x7f673c09aa30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.209+0000 7f67421c8640 1 -- 172.21.15.97:0/1763511330 >> 172.21.15.97:0/1763511330 conn(0x7f673c0d2430 msgr2=0x7f673c0d2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:32.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.209+0000 7f67421c8640 1 -- 172.21.15.97:0/1763511330 shutdown_connections 2024-03-29T04:09:32.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.209+0000 7f67421c8640 1 -- 172.21.15.97:0/1763511330 wait complete. 2024-03-29T04:09:32.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.210+0000 7f67421c8640 1 Processor -- start 2024-03-29T04:09:32.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.210+0000 7f67421c8640 1 -- start start 2024-03-29T04:09:32.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.211+0000 7f67421c8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f673c070990 0x7f673c14f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.211+0000 7f67421c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f673c09af70 0x7f673c0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.211+0000 7f67421c8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c09b930 0x7f673c0c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.211+0000 7f67421c8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f673c070d70 con 0x7f673c09b930 2024-03-29T04:09:32.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.211+0000 7f67421c8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f673c0d1990 con 0x7f673c09af70 2024-03-29T04:09:32.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.211+0000 7f67421c8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f673c070ef0 con 0x7f673c070990 2024-03-29T04:09:32.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.211+0000 7f673bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c09b930 0x7f673c0c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.211+0000 7f673bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c09b930 0x7f673c0c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36126/0 (socket says 172.21.15.97:36126) 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.211+0000 7f673bfff640 1 -- 172.21.15.97:0/403793339 learned_addr learned my addr 172.21.15.97:0/403793339 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.211+0000 7f673affd640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f673c09af70 0x7f673c0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.211+0000 7f673b7fe640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f673c070990 0x7f673c14f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.212+0000 7f673bfff640 1 -- 172.21.15.97:0/403793339 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f673c070990 msgr2=0x7f673c14f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.212+0000 7f673bfff640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f673c070990 0x7f673c14f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.212+0000 7f673bfff640 1 -- 172.21.15.97:0/403793339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f673c09af70 msgr2=0x7f673c0b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.212+0000 7f673bfff640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f673c09af70 0x7f673c0b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.212+0000 7f673bfff640 1 -- 172.21.15.97:0/403793339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c0b5bd0 con 0x7f673c09b930 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.212+0000 7f673bfff640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c09b930 0x7f673c0c7ad0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6734009dd0 tx=0x7f673400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.212+0000 7f6723fff640 1 -- 172.21.15.97:0/403793339 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734018020 con 0x7f673c09b930 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.212+0000 7f6723fff640 1 -- 172.21.15.97:0/403793339 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6734014480 con 0x7f673c09b930 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.212+0000 7f67421c8640 1 -- 172.21.15.97:0/403793339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673c0b5eb0 con 0x7f673c09b930 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.212+0000 7f6723fff640 1 -- 172.21.15.97:0/403793339 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734014870 con 0x7f673c09b930 2024-03-29T04:09:32.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.212+0000 7f67421c8640 1 -- 172.21.15.97:0/403793339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673c0b6ee0 con 0x7f673c09b930 2024-03-29T04:09:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.213+0000 7f67421c8640 1 -- 172.21.15.97:0/403793339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f673c0a05a0 con 0x7f673c09b930 2024-03-29T04:09:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.213+0000 7f6723fff640 1 -- 172.21.15.97:0/403793339 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6734005d10 con 0x7f673c09b930 2024-03-29T04:09:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.214+0000 7f6723fff640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f672806c530 0x7f672806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.214+0000 7f6723fff640 1 -- 172.21.15.97:0/403793339 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7f673408c190 con 0x7f673c09b930 2024-03-29T04:09:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.214+0000 7f673b7fe640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f672806c530 0x7f672806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.214+0000 7f673b7fe640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f672806c530 0x7f672806e9f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f672c001780 tx=0x7f672c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:32.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.218+0000 7f6723fff640 1 -- 172.21.15.97:0/403793339 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6734022070 con 0x7f673c09b930 2024-03-29T04:09:32.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:32.429+0000 7f67421c8640 1 -- 172.21.15.97:0/403793339 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f673c04f9d0 con 0x7f673c09b930 2024-03-29T04:09:33.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.090+0000 7f6723fff640 1 -- 172.21.15.97:0/403793339 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7f6734010040 con 0x7f673c09b930 2024-03-29T04:09:33.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.092+0000 7f67421c8640 1 -- 172.21.15.97:0/403793339 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f672806c530 msgr2=0x7f672806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.092+0000 7f67421c8640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f672806c530 0x7f672806e9f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f672c001780 tx=0x7f672c009040 comp rx=0 tx=0).stop 2024-03-29T04:09:33.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.092+0000 7f67421c8640 1 -- 172.21.15.97:0/403793339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c09b930 msgr2=0x7f673c0c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.092+0000 7f67421c8640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c09b930 0x7f673c0c7ad0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6734009dd0 tx=0x7f673400c490 comp rx=0 tx=0).stop 2024-03-29T04:09:33.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.092+0000 7f67421c8640 1 -- 172.21.15.97:0/403793339 shutdown_connections 2024-03-29T04:09:33.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.092+0000 7f67421c8640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f672806c530 0x7f672806e9f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.092+0000 7f67421c8640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f673c09b930 0x7f673c0c7ad0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.092+0000 7f67421c8640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f673c09af70 0x7f673c0b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.092+0000 7f67421c8640 1 --2- 172.21.15.97:0/403793339 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f673c070990 0x7f673c14f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.092+0000 7f67421c8640 1 -- 172.21.15.97:0/403793339 >> 172.21.15.97:0/403793339 conn(0x7f673c0d2430 msgr2=0x7f673c0b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:33.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.093+0000 7f67421c8640 1 -- 172.21.15.97:0/403793339 shutdown_connections 2024-03-29T04:09:33.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.093+0000 7f67421c8640 1 -- 172.21.15.97:0/403793339 wait complete. 2024-03-29T04:09:33.085 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T04:09:33.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.252+0000 7efee67a8640 1 Processor -- start 2024-03-29T04:09:33.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.252+0000 7efee67a8640 1 -- start start 2024-03-29T04:09:33.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.252+0000 7efee67a8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efee015b350 0x7efee015efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.252+0000 7efee67a8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efee00b4cf0 0x7efee015c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efee67a8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00d0490 0x7efee00d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efee67a8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7efee015bdd0 con 0x7efee00b4cf0 2024-03-29T04:09:33.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efee67a8640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7efee015bad0 con 0x7efee015b350 2024-03-29T04:09:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efee67a8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7efee015f7e0 con 0x7efee00d0490 2024-03-29T04:09:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efedf7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efee00b4cf0 0x7efee015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efedf7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efee00b4cf0 0x7efee015c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36136/0 (socket says 172.21.15.97:36136) 2024-03-29T04:09:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efedf7fe640 1 -- 172.21.15.97:0/2019843969 learned_addr learned my addr 172.21.15.97:0/2019843969 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efee4d45640 1 --2- 172.21.15.97:0/2019843969 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00d0490 0x7efee00d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efedffff640 1 --2- 172.21.15.97:0/2019843969 >> v2:172.21.15.154:3300/0 conn(0x7efee015b350 0x7efee015efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efedf7fe640 1 -- 172.21.15.97:0/2019843969 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00d0490 msgr2=0x7efee00d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efedf7fe640 1 --2- 172.21.15.97:0/2019843969 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00d0490 0x7efee00d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efedf7fe640 1 -- 172.21.15.97:0/2019843969 >> v2:172.21.15.154:3300/0 conn(0x7efee015b350 msgr2=0x7efee015efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efedf7fe640 1 --2- 172.21.15.97:0/2019843969 >> v2:172.21.15.154:3300/0 conn(0x7efee015b350 0x7efee015efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.253+0000 7efedf7fe640 1 -- 172.21.15.97:0/2019843969 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee015fac0 con 0x7efee00b4cf0 2024-03-29T04:09:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.254+0000 7efedf7fe640 1 --2- 172.21.15.97:0/2019843969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee00b4cf0 0x7efee015c260 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efee006f140 tx=0x7efed402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935919cbaff6fa02 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:33.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.254+0000 7efedeffd640 1 -- 172.21.15.97:0/2019843969 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed403b070 con 0x7efee00b4cf0 2024-03-29T04:09:33.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.254+0000 7efedeffd640 1 -- 172.21.15.97:0/2019843969 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed4036440 con 0x7efee00b4cf0 2024-03-29T04:09:33.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.254+0000 7efedeffd640 1 -- 172.21.15.97:0/2019843969 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4036810 con 0x7efee00b4cf0 2024-03-29T04:09:33.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.254+0000 7efee67a8640 1 -- 172.21.15.97:0/2019843969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee00b4cf0 msgr2=0x7efee015c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.254+0000 7efee67a8640 1 --2- 172.21.15.97:0/2019843969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee00b4cf0 0x7efee015c260 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efee006f140 tx=0x7efed402e0e0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.254+0000 7efee67a8640 1 -- 172.21.15.97:0/2019843969 shutdown_connections 2024-03-29T04:09:33.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.254+0000 7efee67a8640 1 --2- 172.21.15.97:0/2019843969 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00d0490 0x7efee00d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.254+0000 7efee67a8640 1 --2- 172.21.15.97:0/2019843969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee00b4cf0 0x7efee015c260 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.254+0000 7efee67a8640 1 --2- 172.21.15.97:0/2019843969 >> v2:172.21.15.154:3300/0 conn(0x7efee015b350 0x7efee015efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.254+0000 7efee67a8640 1 -- 172.21.15.97:0/2019843969 >> 172.21.15.97:0/2019843969 conn(0x7efee006b6f0 msgr2=0x7efee006bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:33.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.254+0000 7efee67a8640 1 -- 172.21.15.97:0/2019843969 shutdown_connections 2024-03-29T04:09:33.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.254+0000 7efee67a8640 1 -- 172.21.15.97:0/2019843969 wait complete. 2024-03-29T04:09:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.256+0000 7efee67a8640 1 Processor -- start 2024-03-29T04:09:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.256+0000 7efee67a8640 1 -- start start 2024-03-29T04:09:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.256+0000 7efee67a8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00b4cf0 0x7efee00b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.256+0000 7efee67a8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee00d0490 0x7efee01595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.256+0000 7efee67a8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efee015b350 0x7efee0150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.256+0000 7efee67a8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efee015b730 con 0x7efee00d0490 2024-03-29T04:09:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.256+0000 7efee67a8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efee015fda0 con 0x7efee015b350 2024-03-29T04:09:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.256+0000 7efee67a8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7efee015b8b0 con 0x7efee00b4cf0 2024-03-29T04:09:33.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.256+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee00d0490 0x7efee01595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee00d0490 0x7efee01595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36140/0 (socket says 172.21.15.97:36140) 2024-03-29T04:09:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efedf7fe640 1 -- 172.21.15.97:0/3057772954 learned_addr learned my addr 172.21.15.97:0/3057772954 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efedffff640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00b4cf0 0x7efee00b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efee4d45640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efee015b350 0x7efee0150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efedf7fe640 1 -- 172.21.15.97:0/3057772954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00b4cf0 msgr2=0x7efee00b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efedf7fe640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00b4cf0 0x7efee00b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efedf7fe640 1 -- 172.21.15.97:0/3057772954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efee015b350 msgr2=0x7efee0150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efedf7fe640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efee015b350 0x7efee0150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efedf7fe640 1 -- 172.21.15.97:0/3057772954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee0158220 con 0x7efee00d0490 2024-03-29T04:09:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efedf7fe640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee00d0490 0x7efee01595a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efed402e120 tx=0x7efed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efedcff9640 1 -- 172.21.15.97:0/3057772954 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4045020 con 0x7efee00d0490 2024-03-29T04:09:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efedcff9640 1 -- 172.21.15.97:0/3057772954 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed40346c0 con 0x7efee00d0490 2024-03-29T04:09:33.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efee67a8640 1 -- 172.21.15.97:0/3057772954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee0158500 con 0x7efee00d0490 2024-03-29T04:09:33.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efedcff9640 1 -- 172.21.15.97:0/3057772954 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed403b040 con 0x7efee00d0490 2024-03-29T04:09:33.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.257+0000 7efee67a8640 1 -- 172.21.15.97:0/3057772954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee0158ac0 con 0x7efee00d0490 2024-03-29T04:09:33.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.258+0000 7efee67a8640 1 -- 172.21.15.97:0/3057772954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee0068640 con 0x7efee00d0490 2024-03-29T04:09:33.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.259+0000 7efedcff9640 1 -- 172.21.15.97:0/3057772954 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efed4048020 con 0x7efee00d0490 2024-03-29T04:09:33.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.259+0000 7efedcff9640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7efecc06c530 0x7efecc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.259+0000 7efedffff640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7efecc06c530 0x7efecc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.259+0000 7efedcff9640 1 -- 172.21.15.97:0/3057772954 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7efed40b00f0 con 0x7efee00d0490 2024-03-29T04:09:33.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.260+0000 7efedffff640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7efecc06c530 0x7efecc06e9f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efed0006fd0 tx=0x7efed0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:33.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.264+0000 7efedcff9640 1 -- 172.21.15.97:0/3057772954 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efed404f030 con 0x7efee00d0490 2024-03-29T04:09:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:33.441+0000 7efee67a8640 1 -- 172.21.15.97:0/3057772954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7efee006bd90 con 0x7efee00d0490 2024-03-29T04:09:34.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.357+0000 7efedcff9640 1 -- 172.21.15.97:0/3057772954 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) ==== 181+0+0 (secure 0 0 0) 0x7efed407e6c0 con 0x7efee00d0490 2024-03-29T04:09:34.337 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-03-29T04:09:34.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.360+0000 7efee67a8640 1 -- 172.21.15.97:0/3057772954 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7efecc06c530 msgr2=0x7efecc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.360+0000 7efee67a8640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7efecc06c530 0x7efecc06e9f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efed0006fd0 tx=0x7efed0008040 comp rx=0 tx=0).stop 2024-03-29T04:09:34.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.360+0000 7efee67a8640 1 -- 172.21.15.97:0/3057772954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee00d0490 msgr2=0x7efee01595a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.360+0000 7efee67a8640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee00d0490 0x7efee01595a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efed402e120 tx=0x7efed40046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.360+0000 7efee67a8640 1 -- 172.21.15.97:0/3057772954 shutdown_connections 2024-03-29T04:09:34.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.360+0000 7efee67a8640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7efecc06c530 0x7efecc06e9f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.360+0000 7efee67a8640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efee015b350 0x7efee0150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.360+0000 7efee67a8640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee00d0490 0x7efee01595a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.360+0000 7efee67a8640 1 --2- 172.21.15.97:0/3057772954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efee00b4cf0 0x7efee00b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.360+0000 7efee67a8640 1 -- 172.21.15.97:0/3057772954 >> 172.21.15.97:0/3057772954 conn(0x7efee006b6f0 msgr2=0x7efee015a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.360+0000 7efee67a8640 1 -- 172.21.15.97:0/3057772954 shutdown_connections 2024-03-29T04:09:34.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.360+0000 7efee67a8640 1 -- 172.21.15.97:0/3057772954 wait complete. 2024-03-29T04:09:34.353 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T04:09:34.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.484+0000 7f1b9ec4a640 1 Processor -- start 2024-03-29T04:09:34.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.484+0000 7f1b9ec4a640 1 -- start start 2024-03-29T04:09:34.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b9ec4a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b9806b510 0x7f1b9806b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b9ec4a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b980cefd0 0x7f1b980cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b9ec4a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b980ce810 con 0x7f1b980b2c10 2024-03-29T04:09:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b9ec4a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9806be30 con 0x7f1b9806b510 2024-03-29T04:09:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b9ec4a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1b980b6420 con 0x7f1b980cefd0 2024-03-29T04:09:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b97fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1b980b2c10 0x7f1b980b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57390/0 (socket says 172.21.15.97:57390) 2024-03-29T04:09:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b97fff640 1 -- 172.21.15.97:0/3916753315 learned_addr learned my addr 172.21.15.97:0/3916753315 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b9d1e7640 1 --2- 172.21.15.97:0/3916753315 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b980cefd0 0x7f1b980cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b9c9e6640 1 --2- 172.21.15.97:0/3916753315 >> v2:172.21.15.154:3300/0 conn(0x7f1b9806b510 0x7f1b9806b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b977fe640 1 -- 172.21.15.97:0/3916753315 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1483097923 0 0) 0x7f1b980ce810 con 0x7f1b980b2c10 2024-03-29T04:09:34.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b977fe640 1 -- 172.21.15.97:0/3916753315 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b88003520 con 0x7f1b980b2c10 2024-03-29T04:09:34.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b9d1e7640 1 -- 172.21.15.97:0/3916753315 >> v2:172.21.15.154:3300/0 conn(0x7f1b9806b510 msgr2=0x7f1b9806b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:34.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b9d1e7640 1 --2- 172.21.15.97:0/3916753315 >> v2:172.21.15.154:3300/0 conn(0x7f1b9806b510 0x7f1b9806b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b9d1e7640 1 -- 172.21.15.97:0/3916753315 >> v1:172.21.15.97:6789/0 conn(0x7f1b980b2c10 legacy=0x7f1b980b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b977fe640 1 -- 172.21.15.97:0/3916753315 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 249264372 0 0) 0x7f1b88003520 con 0x7f1b980b2c10 2024-03-29T04:09:34.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b9d1e7640 1 -- 172.21.15.97:0/3916753315 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b980b6700 con 0x7f1b980cefd0 2024-03-29T04:09:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.485+0000 7f1b9c9e6640 1 --2- 172.21.15.97:0/3916753315 >> v2:172.21.15.154:3300/0 conn(0x7f1b9806b510 0x7f1b9806b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.486+0000 7f1b9d1e7640 1 --2- 172.21.15.97:0/3916753315 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b980cefd0 0x7f1b980cf460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009d90 tx=0x7f1b9002bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1e08618afbd16 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.486+0000 7f1b977fe640 1 -- 172.21.15.97:0/3916753315 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90039070 con 0x7f1b980cefd0 2024-03-29T04:09:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.486+0000 7f1b977fe640 1 -- 172.21.15.97:0/3916753315 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b9002c810 con 0x7f1b980cefd0 2024-03-29T04:09:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.486+0000 7f1b977fe640 1 -- 172.21.15.97:0/3916753315 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84001670 con 0x7f1b980cefd0 2024-03-29T04:09:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.487+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3916753315 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b980cefd0 msgr2=0x7f1b980cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.487+0000 7f1b9ec4a640 1 --2- 172.21.15.97:0/3916753315 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b980cefd0 0x7f1b980cf460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009d90 tx=0x7f1b9002bce0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.487+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3916753315 shutdown_connections 2024-03-29T04:09:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.487+0000 7f1b9ec4a640 1 --2- 172.21.15.97:0/3916753315 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b980cefd0 0x7f1b980cf460 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.487+0000 7f1b9ec4a640 1 --2- 172.21.15.97:0/3916753315 >> v2:172.21.15.154:3300/0 conn(0x7f1b9806b510 0x7f1b9806b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.487+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3916753315 >> 172.21.15.97:0/3916753315 conn(0x7f1b980b6af0 msgr2=0x7f1b9809aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.487+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3916753315 shutdown_connections 2024-03-29T04:09:34.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.487+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3916753315 wait complete. 2024-03-29T04:09:34.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.488+0000 7f1b9ec4a640 1 Processor -- start 2024-03-29T04:09:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.488+0000 7f1b9ec4a640 1 -- start start 2024-03-29T04:09:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9ec4a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9806b510 0x7f1b9809ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9ec4a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b980b2c10 0x7f1b981f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9ec4a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b980cefd0 0x7f1b981f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9ec4a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98145430 con 0x7f1b980cefd0 2024-03-29T04:09:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9ec4a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1b981452b0 con 0x7f1b9806b510 2024-03-29T04:09:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9ec4a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1b980b6040 con 0x7f1b980b2c10 2024-03-29T04:09:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9d1e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b980cefd0 0x7f1b981f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9d1e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b980cefd0 0x7f1b981f3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36146/0 (socket says 172.21.15.97:36146) 2024-03-29T04:09:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9d1e7640 1 -- 172.21.15.97:0/3112464699 learned_addr learned my addr 172.21.15.97:0/3112464699 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b97fff640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b980b2c10 0x7f1b981f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9c9e6640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9806b510 0x7f1b9809ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9d1e7640 1 -- 172.21.15.97:0/3112464699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b980b2c10 msgr2=0x7f1b981f7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9d1e7640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b980b2c10 0x7f1b981f7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9d1e7640 1 -- 172.21.15.97:0/3112464699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9806b510 msgr2=0x7f1b9809ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9d1e7640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9806b510 0x7f1b9809ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.489+0000 7f1b9d1e7640 1 -- 172.21.15.97:0/3112464699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b981f8400 con 0x7f1b980cefd0 2024-03-29T04:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.490+0000 7f1b9d1e7640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b980cefd0 0x7f1b981f3f50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009eb0 tx=0x7f1b90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.490+0000 7f1b957fa640 1 -- 172.21.15.97:0/3112464699 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90039050 con 0x7f1b980cefd0 2024-03-29T04:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.490+0000 7f1b957fa640 1 -- 172.21.15.97:0/3112464699 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b90045070 con 0x7f1b980cefd0 2024-03-29T04:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.490+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3112464699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b981f4960 con 0x7f1b980cefd0 2024-03-29T04:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.490+0000 7f1b957fa640 1 -- 172.21.15.97:0/3112464699 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90009450 con 0x7f1b980cefd0 2024-03-29T04:09:34.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.490+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3112464699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b981f76e0 con 0x7f1b980cefd0 2024-03-29T04:09:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.491+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3112464699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b981f4e80 con 0x7f1b980cefd0 2024-03-29T04:09:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.491+0000 7f1b957fa640 1 -- 172.21.15.97:0/3112464699 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1b90040690 con 0x7f1b980cefd0 2024-03-29T04:09:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.492+0000 7f1b957fa640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b7c06c530 0x7f1b7c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.492+0000 7f1b9c9e6640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b7c06c530 0x7f1b7c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.492+0000 7f1b957fa640 1 -- 172.21.15.97:0/3112464699 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5357+0+0 (secure 0 0 0) 0x7f1b900ada70 con 0x7f1b980cefd0 2024-03-29T04:09:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.492+0000 7f1b9c9e6640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b7c06c530 0x7f1b7c06e9f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009560 tx=0x7f1b8c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.496+0000 7f1b957fa640 1 -- 172.21.15.97:0/3112464699 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1b9004f020 con 0x7f1b980cefd0 2024-03-29T04:09:34.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:34.675+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3112464699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1b98068640 con 0x7f1b980cefd0 2024-03-29T04:09:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.362+0000 7f1b957fa640 1 -- 172.21.15.97:0/3112464699 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) ==== 169+0+0 (secure 0 0 0) 0x7f1b9007bba0 con 0x7f1b980cefd0 2024-03-29T04:09:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-03-29T04:09:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.364+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3112464699 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b7c06c530 msgr2=0x7f1b7c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.364+0000 7f1b9ec4a640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b7c06c530 0x7f1b7c06e9f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009560 tx=0x7f1b8c009190 comp rx=0 tx=0).stop 2024-03-29T04:09:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.364+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3112464699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b980cefd0 msgr2=0x7f1b981f3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.364+0000 7f1b9ec4a640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b980cefd0 0x7f1b981f3f50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009eb0 tx=0x7f1b90004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.364+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3112464699 shutdown_connections 2024-03-29T04:09:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.364+0000 7f1b9ec4a640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1b7c06c530 0x7f1b7c06e9f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.364+0000 7f1b9ec4a640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b980cefd0 0x7f1b981f3f50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.364+0000 7f1b9ec4a640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1b980b2c10 0x7f1b981f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.364+0000 7f1b9ec4a640 1 --2- 172.21.15.97:0/3112464699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b9806b510 0x7f1b9809ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.364+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3112464699 >> 172.21.15.97:0/3112464699 conn(0x7f1b980b6af0 msgr2=0x7f1b980cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.364+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3112464699 shutdown_connections 2024-03-29T04:09:35.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.365+0000 7f1b9ec4a640 1 -- 172.21.15.97:0/3112464699 wait complete. 2024-03-29T04:09:35.359 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-29T04:09:35.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.483+0000 7fb222b8d640 1 Processor -- start 2024-03-29T04:09:35.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.483+0000 7fb222b8d640 1 -- start start 2024-03-29T04:09:35.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.484+0000 7fb222b8d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb21c06f320 0x7fb21c071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.484+0000 7fb222b8d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb21c071f20 con 0x7fb21c15b3d0 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.484+0000 7fb222b8d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb21c072200 con 0x7fb21c0d0410 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.484+0000 7fb222b8d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb21c06f7a0 con 0x7fb21c06f320 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.484+0000 7fb220929640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb21c15b3d0 0x7fb21c0b3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57396/0 (socket says 172.21.15.97:57396) 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.484+0000 7fb220929640 1 -- 172.21.15.97:0/66358533 learned_addr learned my addr 172.21.15.97:0/66358533 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.484+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2710635812 0 0) 0x7fb21c071f20 con 0x7fb21c15b3d0 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.484+0000 7fb21bfff640 1 --2- 172.21.15.97:0/66358533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb21c06f320 0x7fb21c071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.484+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb204003520 con 0x7fb21c15b3d0 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 346438694 0 0) 0x7fb204003520 con 0x7fb21c15b3d0 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb21c06f320 msgr2=0x7fb21c071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21b7fe640 1 --2- 172.21.15.97:0/66358533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb21c06f320 0x7fb21c071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 >> v1:172.21.15.154:6789/0 conn(0x7fb21c0d0410 legacy=0x7fb21c0adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21c0724e0 con 0x7fb21c15b3d0 2024-03-29T04:09:35.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7fb20c001670 con 0x7fb21c15b3d0 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 >> v1:172.21.15.97:6789/0 conn(0x7fb21c15b3d0 legacy=0x7fb21c0b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21b7fe640 1 --2- 172.21.15.97:0/66358533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2040039e0 0x7fb204003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21b7fe640 1 --2- 172.21.15.97:0/66358533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb21c06f320 0x7fb204004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21b7fe640 1 --2- 172.21.15.97:0/66358533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb204004d40 0x7fb2040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c06f7a0 con 0x7fb204004d40 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb20400b9c0 con 0x7fb2040039e0 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb20400ddd0 con 0x7fb21c06f320 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb22112a640 1 --2- 172.21.15.97:0/66358533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb204004d40 0x7fb2040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb220929640 1 --2- 172.21.15.97:0/66358533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2040039e0 0x7fb204003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb21bfff640 1 --2- 172.21.15.97:0/66358533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb21c06f320 0x7fb204004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb22112a640 1 -- 172.21.15.97:0/66358533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb21c06f320 msgr2=0x7fb204004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb22112a640 1 --2- 172.21.15.97:0/66358533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb21c06f320 0x7fb204004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb22112a640 1 -- 172.21.15.97:0/66358533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2040039e0 msgr2=0x7fb204003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:35.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb22112a640 1 --2- 172.21.15.97:0/66358533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2040039e0 0x7fb204003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.485+0000 7fb22112a640 1 -- 172.21.15.97:0/66358533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb204003520 con 0x7fb204004d40 2024-03-29T04:09:35.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.486+0000 7fb22112a640 1 --2- 172.21.15.97:0/66358533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb204004d40 0x7fb2040051c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb214001f60 tx=0x7fb204005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6b3558ff4b5a12 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.486+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214001b30 con 0x7fb204004d40 2024-03-29T04:09:35.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.486+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb21402d710 con 0x7fb204004d40 2024-03-29T04:09:35.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.486+0000 7fb21b7fe640 1 -- 172.21.15.97:0/66358533 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21402db00 con 0x7fb204004d40 2024-03-29T04:09:35.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.487+0000 7fb222b8d640 1 -- 172.21.15.97:0/66358533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb204004d40 msgr2=0x7fb2040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.487+0000 7fb222b8d640 1 --2- 172.21.15.97:0/66358533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb204004d40 0x7fb2040051c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb214001f60 tx=0x7fb204005830 comp rx=0 tx=0).stop 2024-03-29T04:09:35.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.487+0000 7fb22112a640 1 -- 172.21.15.97:0/66358533 reap_dead start 2024-03-29T04:09:35.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.487+0000 7fb222b8d640 1 -- 172.21.15.97:0/66358533 shutdown_connections 2024-03-29T04:09:35.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.487+0000 7fb222b8d640 1 -- 172.21.15.97:0/66358533 >> 172.21.15.97:0/66358533 conn(0x7fb21c06b6f0 msgr2=0x7fb21c06bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.487+0000 7fb222b8d640 1 -- 172.21.15.97:0/66358533 shutdown_connections 2024-03-29T04:09:35.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.487+0000 7fb222b8d640 1 -- 172.21.15.97:0/66358533 wait complete. 2024-03-29T04:09:35.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.488+0000 7fb222b8d640 1 Processor -- start 2024-03-29T04:09:35.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.488+0000 7fb222b8d640 1 -- start start 2024-03-29T04:09:35.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb222b8d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb21c146480 0x7fb21c14d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb222b8d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb21c1f0190 0x7fb21c14f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb222b8d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21c14de80 0x7fb21c14fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb222b8d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c0b3f70 con 0x7fb21c14de80 2024-03-29T04:09:35.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb222b8d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c15b7b0 con 0x7fb21c1f0190 2024-03-29T04:09:35.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb222b8d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fb21c0b40f0 con 0x7fb21c146480 2024-03-29T04:09:35.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb22112a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21c14de80 0x7fb21c14fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb22112a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21c14de80 0x7fb21c14fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36162/0 (socket says 172.21.15.97:36162) 2024-03-29T04:09:35.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb22112a640 1 -- 172.21.15.97:0/4192423717 learned_addr learned my addr 172.21.15.97:0/4192423717 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:35.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb21bfff640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb21c1f0190 0x7fb21c14f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb220929640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb21c146480 0x7fb21c14d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb22112a640 1 -- 172.21.15.97:0/4192423717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb21c146480 msgr2=0x7fb21c14d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb22112a640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb21c146480 0x7fb21c14d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb22112a640 1 -- 172.21.15.97:0/4192423717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb21c1f0190 msgr2=0x7fb21c14f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb22112a640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb21c1f0190 0x7fb21c14f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb22112a640 1 -- 172.21.15.97:0/4192423717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21c14c4f0 con 0x7fb21c14de80 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb22112a640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21c14de80 0x7fb21c14fdc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb214002080 tx=0x7fb2140015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb2197fa640 1 -- 172.21.15.97:0/4192423717 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214001b30 con 0x7fb21c14de80 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb2197fa640 1 -- 172.21.15.97:0/4192423717 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb214046070 con 0x7fb21c14de80 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb222b8d640 1 -- 172.21.15.97:0/4192423717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21c14bc50 con 0x7fb21c14de80 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb2197fa640 1 -- 172.21.15.97:0/4192423717 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21402dd10 con 0x7fb21c14de80 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.489+0000 7fb222b8d640 1 -- 172.21.15.97:0/4192423717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21c14c210 con 0x7fb21c14de80 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.490+0000 7fb222b8d640 1 -- 172.21.15.97:0/4192423717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb21c09e520 con 0x7fb21c14de80 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.491+0000 7fb2197fa640 1 -- 172.21.15.97:0/4192423717 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb21403e040 con 0x7fb21c14de80 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.491+0000 7fb2197fa640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb20806c530 0x7fb20806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.491+0000 7fb220929640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb20806c530 0x7fb20806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.491+0000 7fb2197fa640 1 -- 172.21.15.97:0/4192423717 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fb2140acef0 con 0x7fb21c14de80 2024-03-29T04:09:35.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.492+0000 7fb220929640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb20806c530 0x7fb20806e9f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb20c000c00 tx=0x7fb20c001170 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.496+0000 7fb2197fa640 1 -- 172.21.15.97:0/4192423717 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb21407b8d0 con 0x7fb21c14de80 2024-03-29T04:09:35.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.687+0000 7fb222b8d640 1 -- 172.21.15.97:0/4192423717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb21c0b18e0 con 0x7fb21c14de80 2024-03-29T04:09:35.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.690+0000 7fb2197fa640 1 -- 172.21.15.97:0/4192423717 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fb21407b1d0 con 0x7fb21c14de80 2024-03-29T04:09:35.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.692+0000 7fb222b8d640 1 -- 172.21.15.97:0/4192423717 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb20806c530 msgr2=0x7fb20806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.692+0000 7fb222b8d640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb20806c530 0x7fb20806e9f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb20c000c00 tx=0x7fb20c001170 comp rx=0 tx=0).stop 2024-03-29T04:09:35.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.692+0000 7fb222b8d640 1 -- 172.21.15.97:0/4192423717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21c14de80 msgr2=0x7fb21c14fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.692+0000 7fb222b8d640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21c14de80 0x7fb21c14fdc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb214002080 tx=0x7fb2140015b0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.692+0000 7fb222b8d640 1 -- 172.21.15.97:0/4192423717 shutdown_connections 2024-03-29T04:09:35.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.692+0000 7fb222b8d640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fb20806c530 0x7fb20806e9f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.692+0000 7fb222b8d640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb21c14de80 0x7fb21c14fdc0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.692+0000 7fb222b8d640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb21c1f0190 0x7fb21c14f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.692+0000 7fb222b8d640 1 --2- 172.21.15.97:0/4192423717 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb21c146480 0x7fb21c14d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.693+0000 7fb222b8d640 1 -- 172.21.15.97:0/4192423717 >> 172.21.15.97:0/4192423717 conn(0x7fb21c06b6f0 msgr2=0x7fb21c0ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.693+0000 7fb222b8d640 1 -- 172.21.15.97:0/4192423717 shutdown_connections 2024-03-29T04:09:35.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.693+0000 7fb222b8d640 1 -- 172.21.15.97:0/4192423717 wait complete. 2024-03-29T04:09:35.683 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining' 2024-03-29T04:09:35.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.804+0000 7f352b772640 1 Processor -- start 2024-03-29T04:09:35.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.804+0000 7f352b772640 1 -- start start 2024-03-29T04:09:35.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.805+0000 7f352b772640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f352415e9a0 0x7f3524072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.805+0000 7f352b772640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f352406c520 0x7f352406c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.805+0000 7f352b772640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3524073240 con 0x7f352406e0f0 2024-03-29T04:09:35.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.805+0000 7f352b772640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f352406cec0 con 0x7f352406c520 2024-03-29T04:09:35.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.805+0000 7f352b772640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f352406f2c0 con 0x7f352415e9a0 2024-03-29T04:09:35.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.805+0000 7f3529d0f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f352406c520 0x7f352406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.805+0000 7f3529d0f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f352406c520 0x7f352406c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36172/0 (socket says 172.21.15.97:36172) 2024-03-29T04:09:35.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.805+0000 7f3529d0f640 1 -- 172.21.15.97:0/1031897528 learned_addr learned my addr 172.21.15.97:0/1031897528 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:35.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.805+0000 7f352950e640 1 --2- 172.21.15.97:0/1031897528 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f352415e9a0 0x7f3524072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.805+0000 7f3529d0f640 1 -- 172.21.15.97:0/1031897528 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f352415e9a0 msgr2=0x7f3524072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:35.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.805+0000 7f3529d0f640 1 --2- 172.21.15.97:0/1031897528 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f352415e9a0 0x7f3524072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.805+0000 7f3529d0f640 1 -- 172.21.15.97:0/1031897528 >> v1:172.21.15.154:6789/0 conn(0x7f352406e0f0 legacy=0x7f352406f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:35.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.805+0000 7f3529d0f640 1 -- 172.21.15.97:0/1031897528 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352406e6b0 con 0x7f352406c520 2024-03-29T04:09:35.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.806+0000 7f3529d0f640 1 --2- 172.21.15.97:0/1031897528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352406c520 0x7f352406c980 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f351c009870 tx=0x7f351c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cefe2bcfe70840 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.806+0000 7f351bfff640 1 -- 172.21.15.97:0/1031897528 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c0317c0 con 0x7f352406c520 2024-03-29T04:09:35.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.806+0000 7f351bfff640 1 -- 172.21.15.97:0/1031897528 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f351c037070 con 0x7f352406c520 2024-03-29T04:09:35.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.806+0000 7f351bfff640 1 -- 172.21.15.97:0/1031897528 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c031b80 con 0x7f352406c520 2024-03-29T04:09:35.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.807+0000 7f352b772640 1 -- 172.21.15.97:0/1031897528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352406c520 msgr2=0x7f352406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.807+0000 7f352b772640 1 --2- 172.21.15.97:0/1031897528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352406c520 0x7f352406c980 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f351c009870 tx=0x7f351c02b860 comp rx=0 tx=0).stop 2024-03-29T04:09:35.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.807+0000 7f352b772640 1 -- 172.21.15.97:0/1031897528 shutdown_connections 2024-03-29T04:09:35.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.807+0000 7f352b772640 1 --2- 172.21.15.97:0/1031897528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352406c520 0x7f352406c980 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.807+0000 7f352b772640 1 --2- 172.21.15.97:0/1031897528 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f352415e9a0 0x7f3524072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.807+0000 7f352b772640 1 -- 172.21.15.97:0/1031897528 >> 172.21.15.97:0/1031897528 conn(0x7f3524072090 msgr2=0x7f35240724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.807+0000 7f352b772640 1 -- 172.21.15.97:0/1031897528 shutdown_connections 2024-03-29T04:09:35.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.807+0000 7f352b772640 1 -- 172.21.15.97:0/1031897528 wait complete. 2024-03-29T04:09:35.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.808+0000 7f352b772640 1 Processor -- start 2024-03-29T04:09:35.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.808+0000 7f352b772640 1 -- start start 2024-03-29T04:09:35.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352b772640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352406c520 0x7f352415d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352b772640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f352406e0f0 0x7f35241601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352b772640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f352415e9a0 0x7f3524147170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352b772640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f352406ef50 con 0x7f352406c520 2024-03-29T04:09:35.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352b772640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f352406ec70 con 0x7f352406e0f0 2024-03-29T04:09:35.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352b772640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f352406fe60 con 0x7f352415e9a0 2024-03-29T04:09:35.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352950e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352406c520 0x7f352415d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352950e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352406c520 0x7f352415d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36174/0 (socket says 172.21.15.97:36174) 2024-03-29T04:09:35.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352950e640 1 -- 172.21.15.97:0/2922924967 learned_addr learned my addr 172.21.15.97:0/2922924967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:35.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352950e640 1 -- 172.21.15.97:0/2922924967 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f352415e9a0 msgr2=0x7f3524147170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352950e640 1 --2- 172.21.15.97:0/2922924967 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f352415e9a0 0x7f3524147170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352950e640 1 -- 172.21.15.97:0/2922924967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f352406e0f0 msgr2=0x7f35241601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f3528d0d640 1 --2- 172.21.15.97:0/2922924967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f352406e0f0 0x7f35241601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352950e640 1 --2- 172.21.15.97:0/2922924967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f352406e0f0 0x7f35241601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352950e640 1 -- 172.21.15.97:0/2922924967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3524148f90 con 0x7f352406c520 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f352950e640 1 --2- 172.21.15.97:0/2922924967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352406c520 0x7f352415d7b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3520007e40 tx=0x7f352000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.809+0000 7f3519ffb640 1 -- 172.21.15.97:0/2922924967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3520014620 con 0x7f352406c520 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.810+0000 7f3519ffb640 1 -- 172.21.15.97:0/2922924967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3520019070 con 0x7f352406c520 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.810+0000 7f352b772640 1 -- 172.21.15.97:0/2922924967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3524149550 con 0x7f352406c520 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.810+0000 7f3519ffb640 1 -- 172.21.15.97:0/2922924967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3520014a00 con 0x7f352406c520 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.810+0000 7f352b772640 1 -- 172.21.15.97:0/2922924967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3524149b10 con 0x7f352406c520 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.811+0000 7f352b772640 1 -- 172.21.15.97:0/2922924967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3524068640 con 0x7f352406c520 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.811+0000 7f3519ffb640 1 -- 172.21.15.97:0/2922924967 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3520014ba0 con 0x7f352406c520 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.811+0000 7f3519ffb640 1 --2- 172.21.15.97:0/2922924967 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f350806c530 0x7f350806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.812+0000 7f3528d0d640 1 --2- 172.21.15.97:0/2922924967 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f350806c530 0x7f350806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.812+0000 7f3519ffb640 1 -- 172.21.15.97:0/2922924967 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f352008bbc0 con 0x7f352406c520 2024-03-29T04:09:35.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.812+0000 7f3528d0d640 1 --2- 172.21.15.97:0/2922924967 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f350806c530 0x7f350806e9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f352406cc40 tx=0x7f3514009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:35.816+0000 7f3519ffb640 1 -- 172.21.15.97:0/2922924967 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f352005a570 con 0x7f352406c520 2024-03-29T04:09:36.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.060+0000 7f352b772640 1 -- 172.21.15.97:0/2922924967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]} v 0) -- 0x7f3524149380 con 0x7f352406c520 2024-03-29T04:09:36.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.363+0000 7f3519ffb640 1 -- 172.21.15.97:0/2922924967 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]}]=0 v118) ==== 147+0+0 (secure 0 0 0) 0x7f3520059e70 con 0x7f352406c520 2024-03-29T04:09:36.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.365+0000 7f352b772640 1 -- 172.21.15.97:0/2922924967 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f350806c530 msgr2=0x7f350806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.365+0000 7f352b772640 1 --2- 172.21.15.97:0/2922924967 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f350806c530 0x7f350806e9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f352406cc40 tx=0x7f3514009040 comp rx=0 tx=0).stop 2024-03-29T04:09:36.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.365+0000 7f352b772640 1 -- 172.21.15.97:0/2922924967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352406c520 msgr2=0x7f352415d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.365+0000 7f352b772640 1 --2- 172.21.15.97:0/2922924967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352406c520 0x7f352415d7b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3520007e40 tx=0x7f352000e4b0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.365+0000 7f352b772640 1 -- 172.21.15.97:0/2922924967 shutdown_connections 2024-03-29T04:09:36.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.365+0000 7f352b772640 1 --2- 172.21.15.97:0/2922924967 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f350806c530 0x7f350806e9f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.365+0000 7f352b772640 1 --2- 172.21.15.97:0/2922924967 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f352415e9a0 0x7f3524147170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.365+0000 7f352b772640 1 --2- 172.21.15.97:0/2922924967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f352406e0f0 0x7f35241601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.365+0000 7f352b772640 1 --2- 172.21.15.97:0/2922924967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352406c520 0x7f352415d7b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.365+0000 7f352b772640 1 -- 172.21.15.97:0/2922924967 >> 172.21.15.97:0/2922924967 conn(0x7f3524072090 msgr2=0x7f35241592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.365+0000 7f352b772640 1 -- 172.21.15.97:0/2922924967 shutdown_connections 2024-03-29T04:09:36.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:36.366+0000 7f352b772640 1 -- 172.21.15.97:0/2922924967 wait complete. 2024-03-29T04:09:36.389 INFO:tasks.cephfs_test_runner:test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) ... ERROR 2024-03-29T04:09:36.389 INFO:tasks.cephfs_test_runner: 2024-03-29T04:09:36.390 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-29T04:09:36.390 INFO:tasks.cephfs_test_runner:ERROR: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:09:36.390 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T04:09:36.390 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-29T04:09:36.390 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 322, in test_data_uninlining 2024-03-29T04:09:36.390 INFO:tasks.cephfs_test_runner: self.run_test_worker(opt_clear_cache, 2024-03-29T04:09:36.391 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 259, in run_test_worker 2024-03-29T04:09:36.391 INFO:tasks.cephfs_test_runner: info = self.get_inline_data_info(inodes, files, dir_pins, num_mds) 2024-03-29T04:09:36.391 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 116, in get_inline_data_info 2024-03-29T04:09:36.391 INFO:tasks.cephfs_test_runner: json_out = get_inode_dump(inode, 2024-03-29T04:09:36.391 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 101, in get_inode_dump 2024-03-29T04:09:36.391 INFO:tasks.cephfs_test_runner: json_out = self.fs.rank_tell(['dump', 'inode', inode], rank) 2024-03-29T04:09:36.391 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1332, in rank_tell 2024-03-29T04:09:36.392 INFO:tasks.cephfs_test_runner: return self.mds_tell(*args, mds_id=f"{self.id}:{rank}", **kwargs) 2024-03-29T04:09:36.392 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1319, in mds_tell 2024-03-29T04:09:36.392 INFO:tasks.cephfs_test_runner: out = self.get_ceph_cmd_stdout("tell", f"mds.{mds_id}", *args, **kwargs) 2024-03-29T04:09:36.392 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-03-29T04:09:36.392 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-03-29T04:09:36.392 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-03-29T04:09:36.392 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-03-29T04:09:36.392 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-03-29T04:09:36.392 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-03-29T04:09:36.392 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-29T04:09:36.393 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-29T04:09:36.393 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 447, in run 2024-03-29T04:09:36.393 INFO:tasks.cephfs_test_runner: r = RemoteProcess(client, args, check_status=check_status, hostname=name, 2024-03-29T04:09:36.393 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 65, in __init__ 2024-03-29T04:09:36.393 INFO:tasks.cephfs_test_runner: self.command = quote(args) 2024-03-29T04:09:36.393 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 257, in quote 2024-03-29T04:09:36.393 INFO:tasks.cephfs_test_runner: return ' '.join(_quote(args)) 2024-03-29T04:09:36.393 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 255, in _quote 2024-03-29T04:09:36.394 INFO:tasks.cephfs_test_runner: yield pipes.quote(a) 2024-03-29T04:09:36.394 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/shlex.py", line 325, in quote 2024-03-29T04:09:36.394 INFO:tasks.cephfs_test_runner: if _find_unsafe(s) is None: 2024-03-29T04:09:36.394 INFO:tasks.cephfs_test_runner:TypeError: expected string or bytes-like object 2024-03-29T04:09:36.394 INFO:tasks.cephfs_test_runner: 2024-03-29T04:09:36.394 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T04:09:36.395 INFO:tasks.cephfs_test_runner:Ran 1 test in 89.444s 2024-03-29T04:09:36.395 INFO:tasks.cephfs_test_runner: 2024-03-29T04:09:36.395 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-29T04:09:36.395 INFO:tasks.cephfs_test_runner: 2024-03-29T04:09:36.395 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-29T04:09:36.395 INFO:tasks.cephfs_test_runner:ERROR: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:09:36.396 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T04:09:36.396 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-29T04:09:36.396 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 322, in test_data_uninlining 2024-03-29T04:09:36.396 INFO:tasks.cephfs_test_runner: self.run_test_worker(opt_clear_cache, 2024-03-29T04:09:36.396 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 259, in run_test_worker 2024-03-29T04:09:36.396 INFO:tasks.cephfs_test_runner: info = self.get_inline_data_info(inodes, files, dir_pins, num_mds) 2024-03-29T04:09:36.396 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 116, in get_inline_data_info 2024-03-29T04:09:36.397 INFO:tasks.cephfs_test_runner: json_out = get_inode_dump(inode, 2024-03-29T04:09:36.397 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 101, in get_inode_dump 2024-03-29T04:09:36.397 INFO:tasks.cephfs_test_runner: json_out = self.fs.rank_tell(['dump', 'inode', inode], rank) 2024-03-29T04:09:36.397 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1332, in rank_tell 2024-03-29T04:09:36.397 INFO:tasks.cephfs_test_runner: return self.mds_tell(*args, mds_id=f"{self.id}:{rank}", **kwargs) 2024-03-29T04:09:36.397 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1319, in mds_tell 2024-03-29T04:09:36.397 INFO:tasks.cephfs_test_runner: out = self.get_ceph_cmd_stdout("tell", f"mds.{mds_id}", *args, **kwargs) 2024-03-29T04:09:36.397 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-03-29T04:09:36.398 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-03-29T04:09:36.398 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-03-29T04:09:36.398 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-03-29T04:09:36.398 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-03-29T04:09:36.398 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-03-29T04:09:36.398 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-29T04:09:36.398 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-29T04:09:36.399 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 447, in run 2024-03-29T04:09:36.399 INFO:tasks.cephfs_test_runner: r = RemoteProcess(client, args, check_status=check_status, hostname=name, 2024-03-29T04:09:36.399 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 65, in __init__ 2024-03-29T04:09:36.399 INFO:tasks.cephfs_test_runner: self.command = quote(args) 2024-03-29T04:09:36.399 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 257, in quote 2024-03-29T04:09:36.399 INFO:tasks.cephfs_test_runner: return ' '.join(_quote(args)) 2024-03-29T04:09:36.399 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 255, in _quote 2024-03-29T04:09:36.400 INFO:tasks.cephfs_test_runner: yield pipes.quote(a) 2024-03-29T04:09:36.400 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/shlex.py", line 325, in quote 2024-03-29T04:09:36.400 INFO:tasks.cephfs_test_runner: if _find_unsafe(s) is None: 2024-03-29T04:09:36.400 INFO:tasks.cephfs_test_runner:TypeError: expected string or bytes-like object 2024-03-29T04:09:36.400 INFO:tasks.cephfs_test_runner: 2024-03-29T04:09:36.401 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:09:36.653 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=6cfc6d29007a4ef8880736ffa9fd1789 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:09:36.655 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-29T04:09:36.667 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-29T04:09:36.683 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-29T04:09:36.683 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:09:36.683 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:36.702 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056369,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18829 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T04:09:36.703 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-03-29T04:09:36.703 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:36.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056369,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18829 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T04:09:36.760 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:09:36.760 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:36.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18611 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T04:09:36.779 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-29T04:09:36.779 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:36.837 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18611 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T04:09:36.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:36.838 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-29T04:09:36.854 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T04:09:36.854 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T04:09:36.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:36.854 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:09:36.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:36.922 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:09:36.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:36.990 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-03-29T04:09:37.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:37.087 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-03-29T04:09:37.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:37.102 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-03-29T04:09:37.172 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2024-03-29T04:09:37.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:09:37.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:37.174 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-29T04:09:37.192 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-29T04:09:37.193 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T04:09:37.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:37.193 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:09:37.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:37.265 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:09:37.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:37.335 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2024-03-29T04:09:37.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:37.440 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-29T04:09:37.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:37.455 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2024-03-29T04:09:37.526 INFO:teuthology.orchestra.run.smithi154.stderr:Cannot find device "ceph-brx" 2024-03-29T04:09:37.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:09:37.527 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-29T04:09:37.536 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-29T04:09:37.537 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:09:37.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.660+0000 7fccf904a640 1 Processor -- start 2024-03-29T04:09:37.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.660+0000 7fccf904a640 1 -- start start 2024-03-29T04:09:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.660+0000 7fccf904a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fccf415c670 0x7fccf406c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.660+0000 7fccf904a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fccf40ad940 0x7fccf406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.660+0000 7fccf904a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf4146230 con 0x7fccf406f060 2024-03-29T04:09:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.660+0000 7fccf904a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fccf406dfc0 con 0x7fccf40ad940 2024-03-29T04:09:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.660+0000 7fccf904a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fccf406e2a0 con 0x7fccf415c670 2024-03-29T04:09:37.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.661+0000 7fccf359e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fccf40ad940 0x7fccf406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.661+0000 7fccf359e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fccf40ad940 0x7fccf406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36176/0 (socket says 172.21.15.97:36176) 2024-03-29T04:09:37.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.661+0000 7fccf359e640 1 -- 172.21.15.97:0/312130152 learned_addr learned my addr 172.21.15.97:0/312130152 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:37.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.661+0000 7fccf2d9d640 1 --2- 172.21.15.97:0/312130152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fccf415c670 0x7fccf406c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.661+0000 7fccf359e640 1 -- 172.21.15.97:0/312130152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fccf415c670 msgr2=0x7fccf406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:37.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.661+0000 7fccf359e640 1 --2- 172.21.15.97:0/312130152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fccf415c670 0x7fccf406c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.661+0000 7fccf359e640 1 -- 172.21.15.97:0/312130152 >> v1:172.21.15.154:6789/0 conn(0x7fccf406f060 legacy=0x7fccf40ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.661+0000 7fccf359e640 1 -- 172.21.15.97:0/312130152 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf406e580 con 0x7fccf40ad940 2024-03-29T04:09:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.661+0000 7fccf2d9d640 1 --2- 172.21.15.97:0/312130152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fccf415c670 0x7fccf406c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.662+0000 7fccf359e640 1 --2- 172.21.15.97:0/312130152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf40ad940 0x7fccf406d930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fccec009e50 tx=0x7fccec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e705afc0b0a45d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.662+0000 7fccf259c640 1 -- 172.21.15.97:0/312130152 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec02c9e0 con 0x7fccf40ad940 2024-03-29T04:09:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.662+0000 7fccf259c640 1 -- 172.21.15.97:0/312130152 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccec02cb80 con 0x7fccf40ad940 2024-03-29T04:09:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.662+0000 7fccf259c640 1 -- 172.21.15.97:0/312130152 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec0343f0 con 0x7fccf40ad940 2024-03-29T04:09:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.662+0000 7fccf904a640 1 -- 172.21.15.97:0/312130152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf40ad940 msgr2=0x7fccf406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.662+0000 7fccf904a640 1 --2- 172.21.15.97:0/312130152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf40ad940 0x7fccf406d930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fccec009e50 tx=0x7fccec02be40 comp rx=0 tx=0).stop 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.662+0000 7fccf904a640 1 -- 172.21.15.97:0/312130152 shutdown_connections 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.662+0000 7fccf904a640 1 --2- 172.21.15.97:0/312130152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf40ad940 0x7fccf406d930 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.662+0000 7fccf904a640 1 --2- 172.21.15.97:0/312130152 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fccf415c670 0x7fccf406c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.662+0000 7fccf904a640 1 -- 172.21.15.97:0/312130152 >> 172.21.15.97:0/312130152 conn(0x7fccf406b510 msgr2=0x7fccf406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.662+0000 7fccf904a640 1 -- 172.21.15.97:0/312130152 shutdown_connections 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.662+0000 7fccf904a640 1 -- 172.21.15.97:0/312130152 wait complete. 2024-03-29T04:09:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.664+0000 7fccf904a640 1 Processor -- start 2024-03-29T04:09:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.664+0000 7fccf904a640 1 -- start start 2024-03-29T04:09:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.664+0000 7fccf904a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf406f060 0x7fccf40b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.664+0000 7fccf904a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccf40ad940 0x7fccf41f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.664+0000 7fccf904a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fccf415c670 0x7fccf41441c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf904a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fccf40b32c0 con 0x7fccf406f060 2024-03-29T04:09:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf904a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fccf415ca50 con 0x7fccf40ad940 2024-03-29T04:09:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf904a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fccf40b3440 con 0x7fccf415c670 2024-03-29T04:09:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf406f060 0x7fccf40b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf406f060 0x7fccf40b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36192/0 (socket says 172.21.15.97:36192) 2024-03-29T04:09:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf2d9d640 1 -- 172.21.15.97:0/3296629562 learned_addr learned my addr 172.21.15.97:0/3296629562 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf359e640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fccf415c670 0x7fccf41441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccea59c640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccf40ad940 0x7fccf41f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf2d9d640 1 -- 172.21.15.97:0/3296629562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fccf415c670 msgr2=0x7fccf41441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf2d9d640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fccf415c670 0x7fccf41441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf2d9d640 1 -- 172.21.15.97:0/3296629562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccf40ad940 msgr2=0x7fccf41f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf2d9d640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccf40ad940 0x7fccf41f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf2d9d640 1 -- 172.21.15.97:0/3296629562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf41f2df0 con 0x7fccf406f060 2024-03-29T04:09:37.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf2d9d640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf406f060 0x7fccf40b1090 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c820 tx=0x7fcce400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccebfff640 1 -- 172.21.15.97:0/3296629562 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4015070 con 0x7fccf406f060 2024-03-29T04:09:37.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccebfff640 1 -- 172.21.15.97:0/3296629562 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce4007a30 con 0x7fccf406f060 2024-03-29T04:09:37.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf904a640 1 -- 172.21.15.97:0/3296629562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf41f3430 con 0x7fccf406f060 2024-03-29T04:09:37.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccebfff640 1 -- 172.21.15.97:0/3296629562 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4007de0 con 0x7fccf406f060 2024-03-29T04:09:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.665+0000 7fccf904a640 1 -- 172.21.15.97:0/3296629562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf4145ab0 con 0x7fccf406f060 2024-03-29T04:09:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.666+0000 7fccf904a640 1 -- 172.21.15.97:0/3296629562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf4068640 con 0x7fccf406f060 2024-03-29T04:09:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.667+0000 7fccebfff640 1 -- 172.21.15.97:0/3296629562 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcce4018070 con 0x7fccf406f060 2024-03-29T04:09:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.667+0000 7fccebfff640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fccd406c560 0x7fccd406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.667+0000 7fccebfff640 1 -- 172.21.15.97:0/3296629562 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fcce408b270 con 0x7fccf406f060 2024-03-29T04:09:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.667+0000 7fccea59c640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fccd406c560 0x7fccd406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.668+0000 7fccea59c640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fccd406c560 0x7fccd406ea20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fccf40b4e10 tx=0x7fccdc007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.672+0000 7fccebfff640 1 -- 172.21.15.97:0/3296629562 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcce40594c0 con 0x7fccf406f060 2024-03-29T04:09:37.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.845+0000 7fccf904a640 1 -- 172.21.15.97:0/3296629562 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fccf415f490 con 0x7fccd406c560 2024-03-29T04:09:37.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.845+0000 7fccebfff640 1 -- 172.21.15.97:0/3296629562 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13491 (secure 0 0 0) 0x7fccf415f490 con 0x7fccd406c560 2024-03-29T04:09:37.825 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:37.826 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-29T04:09:37.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.848+0000 7fccf904a640 1 -- 172.21.15.97:0/3296629562 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fccd406c560 msgr2=0x7fccd406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.848+0000 7fccf904a640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fccd406c560 0x7fccd406ea20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fccf40b4e10 tx=0x7fccdc007e30 comp rx=0 tx=0).stop 2024-03-29T04:09:37.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.848+0000 7fccf904a640 1 -- 172.21.15.97:0/3296629562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf406f060 msgr2=0x7fccf40b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.848+0000 7fccf904a640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf406f060 0x7fccf40b1090 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c820 tx=0x7fcce400cce0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.848+0000 7fccf904a640 1 -- 172.21.15.97:0/3296629562 shutdown_connections 2024-03-29T04:09:37.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.848+0000 7fccf904a640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fccd406c560 0x7fccd406ea20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.848+0000 7fccf904a640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fccf415c670 0x7fccf41441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.848+0000 7fccf904a640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccf40ad940 0x7fccf41f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.848+0000 7fccf904a640 1 --2- 172.21.15.97:0/3296629562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccf406f060 0x7fccf40b1090 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.848+0000 7fccf904a640 1 -- 172.21.15.97:0/3296629562 >> 172.21.15.97:0/3296629562 conn(0x7fccf406b510 msgr2=0x7fccf40b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.848+0000 7fccf904a640 1 -- 172.21.15.97:0/3296629562 shutdown_connections 2024-03-29T04:09:37.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.848+0000 7fccf904a640 1 -- 172.21.15.97:0/3296629562 wait complete. 2024-03-29T04:09:37.838 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-03-29T04:09:36.659045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":561504,"kb_used_data":345368,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754413216,"statfs":{"total":773094113280,"available":772519133184,"internally_reserved":0,"allocated":353656832,"data_stored":351456608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-174569470,"num_objects":-138,"num_object_clones":0,"num_object_copies":-276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-138,"num_whiteouts":0,"num_read":-77,"num_read_kb":-2,"num_write":-353,"num_write_kb":-170625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000920"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":193,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:32.803857+0000","last_change":"2024-03-29T04:07:29.856799+0000","last_active":"2024-03-29T04:08:32.803857+0000","last_peered":"2024-03-29T04:08:32.803857+0000","last_clean":"2024-03-29T04:08:32.803857+0000","last_became_active":"2024-03-29T04:07:29.855873+0000","last_became_peered":"2024-03-29T04:07:29.855873+0000","last_unstale":"2024-03-29T04:08:32.803857+0000","last_undegraded":"2024-03-29T04:08:32.803857+0000","last_fullsized":"2024-03-29T04:08:32.803857+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_clean_scrub_stamp":"2024-03-29T04:07:28.661516+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:25:15.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73416,"kb_used_data":46400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94298424,"statfs":{"total":96636764160,"available":96561586176,"internally_reserved":0,"allocated":47513600,"data_stored":47237297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52092,"kb_used_data":25072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94319748,"statfs":{"total":96636764160,"available":96583421952,"internally_reserved":0,"allocated":25673728,"data_stored":25418645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89084,"kb_used_data":62068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94282756,"statfs":{"total":96636764160,"available":96545542144,"internally_reserved":0,"allocated":63557632,"data_stored":63282559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85744,"kb_used_data":58724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94286096,"statfs":{"total":96636764160,"available":96548962304,"internally_reserved":0,"allocated":60133376,"data_stored":59855078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65532,"kb_used_data":38516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94306308,"statfs":{"total":96636764160,"available":96569659392,"internally_reserved":0,"allocated":39440384,"data_stored":39145402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52304,"kb_used_data":25288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94319536,"statfs":{"total":96636764160,"available":96583204864,"internally_reserved":0,"allocated":25894912,"data_stored":25632306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82300,"kb_used_data":55284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94289540,"statfs":{"total":96636764160,"available":96552488960,"internally_reserved":0,"allocated":56610816,"data_stored":56319972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61032,"kb_used_data":34016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94310808,"statfs":{"total":96636764160,"available":96574267392,"internally_reserved":0,"allocated":34832384,"data_stored":34565349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:09:37.839 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:09:37.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.959+0000 7f970eac5640 1 Processor -- start 2024-03-29T04:09:37.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.959+0000 7f970eac5640 1 -- start start 2024-03-29T04:09:37.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.960+0000 7f970eac5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97080b3c50 0x7f97080d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.960+0000 7f970eac5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f97080d3140 0x7f970806cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.960+0000 7f970eac5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97080cfb30 con 0x7f970815e930 2024-03-29T04:09:37.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.960+0000 7f970eac5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f970806c780 con 0x7f97080d3140 2024-03-29T04:09:37.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.960+0000 7f970eac5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f97080cfe10 con 0x7f97080b3c50 2024-03-29T04:09:37.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.960+0000 7f970d062640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f97080d3140 0x7f970806cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.960+0000 7f970d062640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f97080d3140 0x7f970806cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36208/0 (socket says 172.21.15.97:36208) 2024-03-29T04:09:37.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.960+0000 7f970d062640 1 -- 172.21.15.97:0/3318187968 learned_addr learned my addr 172.21.15.97:0/3318187968 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:37.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.961+0000 7f970d062640 1 -- 172.21.15.97:0/3318187968 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97080b3c50 msgr2=0x7f97080d2c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:09:37.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.961+0000 7f970d062640 1 --2- 172.21.15.97:0/3318187968 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97080b3c50 0x7f97080d2c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.961+0000 7f970d062640 1 -- 172.21.15.97:0/3318187968 >> v1:172.21.15.154:6789/0 conn(0x7f970815e930 legacy=0x7f97080cf130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:09:37.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.961+0000 7f970d062640 1 -- 172.21.15.97:0/3318187968 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97080d00f0 con 0x7f97080d3140 2024-03-29T04:09:37.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.961+0000 7f970d062640 1 --2- 172.21.15.97:0/3318187968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97080d3140 0x7f970806cc10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9700009e50 tx=0x7f970002be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d969c0a3f6290ae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.962+0000 7f97077fe640 1 -- 172.21.15.97:0/3318187968 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970002ca10 con 0x7f97080d3140 2024-03-29T04:09:37.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.962+0000 7f97077fe640 1 -- 172.21.15.97:0/3318187968 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970002cbb0 con 0x7f97080d3140 2024-03-29T04:09:37.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.962+0000 7f97077fe640 1 -- 172.21.15.97:0/3318187968 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000343f0 con 0x7f97080d3140 2024-03-29T04:09:37.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.962+0000 7f970eac5640 1 -- 172.21.15.97:0/3318187968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97080d3140 msgr2=0x7f970806cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.962+0000 7f970eac5640 1 --2- 172.21.15.97:0/3318187968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97080d3140 0x7f970806cc10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9700009e50 tx=0x7f970002be70 comp rx=0 tx=0).stop 2024-03-29T04:09:37.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.962+0000 7f970eac5640 1 -- 172.21.15.97:0/3318187968 shutdown_connections 2024-03-29T04:09:37.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.962+0000 7f970eac5640 1 --2- 172.21.15.97:0/3318187968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97080d3140 0x7f970806cc10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.962+0000 7f970eac5640 1 --2- 172.21.15.97:0/3318187968 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97080b3c50 0x7f97080d2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.962+0000 7f970eac5640 1 -- 172.21.15.97:0/3318187968 >> 172.21.15.97:0/3318187968 conn(0x7f970808bfc0 msgr2=0x7f97080d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.962+0000 7f970eac5640 1 -- 172.21.15.97:0/3318187968 shutdown_connections 2024-03-29T04:09:37.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.962+0000 7f970eac5640 1 -- 172.21.15.97:0/3318187968 wait complete. 2024-03-29T04:09:37.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.963+0000 7f970eac5640 1 Processor -- start 2024-03-29T04:09:37.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.963+0000 7f970eac5640 1 -- start start 2024-03-29T04:09:37.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f970eac5640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97080b3c50 0x7f970814a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f970eac5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97080d3140 0x7f97080b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f970eac5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f970815e930 0x7f97080b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f970eac5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f970815ed10 con 0x7f97080d3140 2024-03-29T04:09:37.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f970eac5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97080cf840 con 0x7f970815e930 2024-03-29T04:09:37.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f970eac5640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f970815ee90 con 0x7f97080b3c50 2024-03-29T04:09:37.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f9707fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97080d3140 0x7f97080b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f9707fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97080d3140 0x7f97080b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36224/0 (socket says 172.21.15.97:36224) 2024-03-29T04:09:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f9707fff640 1 -- 172.21.15.97:0/2476289082 learned_addr learned my addr 172.21.15.97:0/2476289082 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f9707fff640 1 -- 172.21.15.97:0/2476289082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97080b3c50 msgr2=0x7f970814a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f9707fff640 1 --2- 172.21.15.97:0/2476289082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97080b3c50 0x7f970814a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f9707fff640 1 -- 172.21.15.97:0/2476289082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f970815e930 msgr2=0x7f97080b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f9707fff640 1 --2- 172.21.15.97:0/2476289082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f970815e930 0x7f97080b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.964+0000 7f9707fff640 1 -- 172.21.15.97:0/2476289082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97081614b0 con 0x7f97080d3140 2024-03-29T04:09:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.965+0000 7f9707fff640 1 --2- 172.21.15.97:0/2476289082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97080d3140 0x7f97080b13c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00b470 tx=0x7f96fc00b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.965+0000 7f97057fa640 1 -- 172.21.15.97:0/2476289082 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc015020 con 0x7f97080d3140 2024-03-29T04:09:37.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.965+0000 7f97057fa640 1 -- 172.21.15.97:0/2476289082 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96fc011890 con 0x7f97080d3140 2024-03-29T04:09:37.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.965+0000 7f970eac5640 1 -- 172.21.15.97:0/2476289082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9708161790 con 0x7f97080d3140 2024-03-29T04:09:37.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.965+0000 7f97057fa640 1 -- 172.21.15.97:0/2476289082 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc011c80 con 0x7f97080d3140 2024-03-29T04:09:37.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.965+0000 7f970eac5640 1 -- 172.21.15.97:0/2476289082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9708161d50 con 0x7f97080d3140 2024-03-29T04:09:37.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.966+0000 7f970eac5640 1 -- 172.21.15.97:0/2476289082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9708068640 con 0x7f97080d3140 2024-03-29T04:09:37.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.966+0000 7f97057fa640 1 -- 172.21.15.97:0/2476289082 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f96fc018020 con 0x7f97080d3140 2024-03-29T04:09:37.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.966+0000 7f97057fa640 1 --2- 172.21.15.97:0/2476289082 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f96f406c530 0x7f96f406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.967+0000 7f97057fa640 1 -- 172.21.15.97:0/2476289082 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f96fc08b530 con 0x7f97080d3140 2024-03-29T04:09:37.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.967+0000 7f970c861640 1 --2- 172.21.15.97:0/2476289082 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f96f406c530 0x7f96f406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.967+0000 7f970c861640 1 --2- 172.21.15.97:0/2476289082 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f96f406c530 0x7f96f406e9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009650 tx=0x7f96f8006ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:37.971+0000 7f97057fa640 1 -- 172.21.15.97:0/2476289082 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f96fc01f020 con 0x7f97080d3140 2024-03-29T04:09:38.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.142+0000 7f970eac5640 1 -- 172.21.15.97:0/2476289082 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f970814ac60 con 0x7f96f406c530 2024-03-29T04:09:38.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.143+0000 7f97057fa640 1 -- 172.21.15.97:0/2476289082 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13491 (secure 0 0 0) 0x7f970814ac60 con 0x7f96f406c530 2024-03-29T04:09:38.122 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:38.123 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-29T04:09:38.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.144+0000 7f970eac5640 1 -- 172.21.15.97:0/2476289082 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f96f406c530 msgr2=0x7f96f406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.144+0000 7f970eac5640 1 --2- 172.21.15.97:0/2476289082 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f96f406c530 0x7f96f406e9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009650 tx=0x7f96f8006ba0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.145+0000 7f970eac5640 1 -- 172.21.15.97:0/2476289082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97080d3140 msgr2=0x7f97080b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.145+0000 7f970eac5640 1 --2- 172.21.15.97:0/2476289082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97080d3140 0x7f97080b13c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00b470 tx=0x7f96fc00b930 comp rx=0 tx=0).stop 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.145+0000 7f970eac5640 1 -- 172.21.15.97:0/2476289082 shutdown_connections 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.145+0000 7f970eac5640 1 --2- 172.21.15.97:0/2476289082 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f96f406c530 0x7f96f406e9f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.145+0000 7f970eac5640 1 --2- 172.21.15.97:0/2476289082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f970815e930 0x7f97080b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.145+0000 7f970eac5640 1 --2- 172.21.15.97:0/2476289082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97080d3140 0x7f97080b13c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.145+0000 7f970eac5640 1 --2- 172.21.15.97:0/2476289082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97080b3c50 0x7f970814a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.145+0000 7f970eac5640 1 -- 172.21.15.97:0/2476289082 >> 172.21.15.97:0/2476289082 conn(0x7f970808bfc0 msgr2=0x7f97080d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:38.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.145+0000 7f970eac5640 1 -- 172.21.15.97:0/2476289082 shutdown_connections 2024-03-29T04:09:38.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.145+0000 7f970eac5640 1 -- 172.21.15.97:0/2476289082 wait complete. 2024-03-29T04:09:38.135 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-03-29T04:09:36.659045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":561504,"kb_used_data":345368,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754413216,"statfs":{"total":773094113280,"available":772519133184,"internally_reserved":0,"allocated":353656832,"data_stored":351456608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-174569470,"num_objects":-138,"num_object_clones":0,"num_object_copies":-276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-138,"num_whiteouts":0,"num_read":-77,"num_read_kb":-2,"num_write":-353,"num_write_kb":-170625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000920"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":193,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:32.803857+0000","last_change":"2024-03-29T04:07:29.856799+0000","last_active":"2024-03-29T04:08:32.803857+0000","last_peered":"2024-03-29T04:08:32.803857+0000","last_clean":"2024-03-29T04:08:32.803857+0000","last_became_active":"2024-03-29T04:07:29.855873+0000","last_became_peered":"2024-03-29T04:07:29.855873+0000","last_unstale":"2024-03-29T04:08:32.803857+0000","last_undegraded":"2024-03-29T04:08:32.803857+0000","last_fullsized":"2024-03-29T04:08:32.803857+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_clean_scrub_stamp":"2024-03-29T04:07:28.661516+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:25:15.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73416,"kb_used_data":46400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94298424,"statfs":{"total":96636764160,"available":96561586176,"internally_reserved":0,"allocated":47513600,"data_stored":47237297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52092,"kb_used_data":25072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94319748,"statfs":{"total":96636764160,"available":96583421952,"internally_reserved":0,"allocated":25673728,"data_stored":25418645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89084,"kb_used_data":62068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94282756,"statfs":{"total":96636764160,"available":96545542144,"internally_reserved":0,"allocated":63557632,"data_stored":63282559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85744,"kb_used_data":58724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94286096,"statfs":{"total":96636764160,"available":96548962304,"internally_reserved":0,"allocated":60133376,"data_stored":59855078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65532,"kb_used_data":38516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94306308,"statfs":{"total":96636764160,"available":96569659392,"internally_reserved":0,"allocated":39440384,"data_stored":39145402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52304,"kb_used_data":25288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94319536,"statfs":{"total":96636764160,"available":96583204864,"internally_reserved":0,"allocated":25894912,"data_stored":25632306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82300,"kb_used_data":55284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94289540,"statfs":{"total":96636764160,"available":96552488960,"internally_reserved":0,"allocated":56610816,"data_stored":56319972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61032,"kb_used_data":34016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94310808,"statfs":{"total":96636764160,"available":96574267392,"internally_reserved":0,"allocated":34832384,"data_stored":34565349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:09:38.135 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-29T04:09:38.136 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:09:38.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.256+0000 7f325d909640 1 Processor -- start 2024-03-29T04:09:38.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.256+0000 7f325d909640 1 -- start start 2024-03-29T04:09:38.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f325d909640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f325815b370 0x7f32580b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f325d909640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3258073290 0x7f325815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f325d909640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3258071ae0 con 0x7f325806f3f0 2024-03-29T04:09:38.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f325d909640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f3258071dc0 con 0x7f325815b370 2024-03-29T04:09:38.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f325d909640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f32580720a0 con 0x7f3258073290 2024-03-29T04:09:38.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f3256ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f325815b370 0x7f32580b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f3256ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f325815b370 0x7f32580b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36240/0 (socket says 172.21.15.97:36240) 2024-03-29T04:09:38.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f3256ffd640 1 -- 172.21.15.97:0/1963566981 learned_addr learned my addr 172.21.15.97:0/1963566981 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:38.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f3256ffd640 1 -- 172.21.15.97:0/1963566981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3258073290 msgr2=0x7f325815f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:38.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f32577fe640 1 --2- 172.21.15.97:0/1963566981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3258073290 0x7f325815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f3256ffd640 1 --2- 172.21.15.97:0/1963566981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3258073290 0x7f325815f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f3256ffd640 1 -- 172.21.15.97:0/1963566981 >> v1:172.21.15.154:6789/0 conn(0x7f325806f3f0 legacy=0x7f3258072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:38.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f3256ffd640 1 -- 172.21.15.97:0/1963566981 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3258072380 con 0x7f325815b370 2024-03-29T04:09:38.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.257+0000 7f32577fe640 1 --2- 172.21.15.97:0/1963566981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3258073290 0x7f325815f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:38.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.258+0000 7f3256ffd640 1 --2- 172.21.15.97:0/1963566981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f325815b370 0x7f32580b32c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f324800d150 tx=0x7f324802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd12d18288bf96a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.258+0000 7f3255ffb640 1 -- 172.21.15.97:0/1963566981 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324802ea80 con 0x7f325815b370 2024-03-29T04:09:38.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.258+0000 7f3255ffb640 1 -- 172.21.15.97:0/1963566981 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f324802ec20 con 0x7f325815b370 2024-03-29T04:09:38.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.258+0000 7f3255ffb640 1 -- 172.21.15.97:0/1963566981 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248009600 con 0x7f325815b370 2024-03-29T04:09:38.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.259+0000 7f325d909640 1 -- 172.21.15.97:0/1963566981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f325815b370 msgr2=0x7f32580b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.259+0000 7f325d909640 1 --2- 172.21.15.97:0/1963566981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f325815b370 0x7f32580b32c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f324800d150 tx=0x7f324802e0e0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.259+0000 7f325d909640 1 -- 172.21.15.97:0/1963566981 shutdown_connections 2024-03-29T04:09:38.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.259+0000 7f325d909640 1 --2- 172.21.15.97:0/1963566981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3258073290 0x7f325815f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.259+0000 7f325d909640 1 --2- 172.21.15.97:0/1963566981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f325815b370 0x7f32580b32c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.259+0000 7f325d909640 1 -- 172.21.15.97:0/1963566981 >> 172.21.15.97:0/1963566981 conn(0x7f325806b510 msgr2=0x7f325806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:38.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.259+0000 7f325d909640 1 -- 172.21.15.97:0/1963566981 shutdown_connections 2024-03-29T04:09:38.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.259+0000 7f325d909640 1 -- 172.21.15.97:0/1963566981 wait complete. 2024-03-29T04:09:38.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.260+0000 7f325d909640 1 Processor -- start 2024-03-29T04:09:38.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.260+0000 7f325d909640 1 -- start start 2024-03-29T04:09:38.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f325d909640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f325806f3f0 0x7f325809ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f325d909640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3258073290 0x7f3258149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f325d909640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f325815b370 0x7f32580cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f325d909640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f32580b3980 con 0x7f3258073290 2024-03-29T04:09:38.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f325d909640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f32580b3800 con 0x7f325806f3f0 2024-03-29T04:09:38.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f325d909640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f325815b750 con 0x7f325815b370 2024-03-29T04:09:38.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f32567fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3258073290 0x7f3258149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f32567fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3258073290 0x7f3258149590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36252/0 (socket says 172.21.15.97:36252) 2024-03-29T04:09:38.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f32567fc640 1 -- 172.21.15.97:0/87609672 learned_addr learned my addr 172.21.15.97:0/87609672 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:38.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f32577fe640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f325815b370 0x7f32580cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f3256ffd640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f325806f3f0 0x7f325809ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f32567fc640 1 -- 172.21.15.97:0/87609672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f325815b370 msgr2=0x7f32580cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f32567fc640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f325815b370 0x7f32580cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f32567fc640 1 -- 172.21.15.97:0/87609672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f325806f3f0 msgr2=0x7f325809ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f32567fc640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f325806f3f0 0x7f325809ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f32567fc640 1 -- 172.21.15.97:0/87609672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325809d1a0 con 0x7f3258073290 2024-03-29T04:09:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f32577fe640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f325815b370 0x7f32580cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f3256ffd640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f325806f3f0 0x7f325809ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.261+0000 7f32567fc640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3258073290 0x7f3258149590 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f324c00c430 tx=0x7f324c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.262+0000 7f323ffff640 1 -- 172.21.15.97:0/87609672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c011790 con 0x7f3258073290 2024-03-29T04:09:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.262+0000 7f323ffff640 1 -- 172.21.15.97:0/87609672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f324c017070 con 0x7f3258073290 2024-03-29T04:09:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.262+0000 7f325d909640 1 -- 172.21.15.97:0/87609672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f325809d480 con 0x7f3258073290 2024-03-29T04:09:38.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.262+0000 7f323ffff640 1 -- 172.21.15.97:0/87609672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c011bb0 con 0x7f3258073290 2024-03-29T04:09:38.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.262+0000 7f325d909640 1 -- 172.21.15.97:0/87609672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f325809da40 con 0x7f3258073290 2024-03-29T04:09:38.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.263+0000 7f325d909640 1 -- 172.21.15.97:0/87609672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3258068640 con 0x7f3258073290 2024-03-29T04:09:38.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.263+0000 7f323ffff640 1 -- 172.21.15.97:0/87609672 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f324c01c460 con 0x7f3258073290 2024-03-29T04:09:38.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.263+0000 7f323ffff640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f324406c530 0x7f324406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.264+0000 7f323ffff640 1 -- 172.21.15.97:0/87609672 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f324c08b370 con 0x7f3258073290 2024-03-29T04:09:38.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.264+0000 7f3256ffd640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f324406c530 0x7f324406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.264+0000 7f3256ffd640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f324406c530 0x7f324406e9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3248037000 tx=0x7f3248032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.268+0000 7f323ffff640 1 -- 172.21.15.97:0/87609672 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f324c059c00 con 0x7f3258073290 2024-03-29T04:09:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.441+0000 7f325d909640 1 -- 172.21.15.97:0/87609672 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f32580d0000 con 0x7f324406c530 2024-03-29T04:09:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.442+0000 7f323ffff640 1 -- 172.21.15.97:0/87609672 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13491 (secure 0 0 0) 0x7f32580d0000 con 0x7f324406c530 2024-03-29T04:09:38.422 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:38.422 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-29T04:09:38.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.444+0000 7f325d909640 1 -- 172.21.15.97:0/87609672 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f324406c530 msgr2=0x7f324406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.444+0000 7f325d909640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f324406c530 0x7f324406e9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3248037000 tx=0x7f3248032000 comp rx=0 tx=0).stop 2024-03-29T04:09:38.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.444+0000 7f325d909640 1 -- 172.21.15.97:0/87609672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3258073290 msgr2=0x7f3258149590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.444+0000 7f325d909640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3258073290 0x7f3258149590 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f324c00c430 tx=0x7f324c00c8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.444+0000 7f325d909640 1 -- 172.21.15.97:0/87609672 shutdown_connections 2024-03-29T04:09:38.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.444+0000 7f325d909640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f324406c530 0x7f324406e9f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.444+0000 7f325d909640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f325815b370 0x7f32580cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.444+0000 7f325d909640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3258073290 0x7f3258149590 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.444+0000 7f325d909640 1 --2- 172.21.15.97:0/87609672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f325806f3f0 0x7f325809ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.444+0000 7f325d909640 1 -- 172.21.15.97:0/87609672 >> 172.21.15.97:0/87609672 conn(0x7f325806b510 msgr2=0x7f32580b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:38.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.444+0000 7f325d909640 1 -- 172.21.15.97:0/87609672 shutdown_connections 2024-03-29T04:09:38.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.444+0000 7f325d909640 1 -- 172.21.15.97:0/87609672 wait complete. 2024-03-29T04:09:38.434 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-03-29T04:09:36.659045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":561504,"kb_used_data":345368,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754413216,"statfs":{"total":773094113280,"available":772519133184,"internally_reserved":0,"allocated":353656832,"data_stored":351456608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-174569470,"num_objects":-138,"num_object_clones":0,"num_object_copies":-276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-138,"num_whiteouts":0,"num_read":-77,"num_read_kb":-2,"num_write":-353,"num_write_kb":-170625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000920"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":193,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-29T04:08:32.803857+0000","last_change":"2024-03-29T04:07:29.856799+0000","last_active":"2024-03-29T04:08:32.803857+0000","last_peered":"2024-03-29T04:08:32.803857+0000","last_clean":"2024-03-29T04:08:32.803857+0000","last_became_active":"2024-03-29T04:07:29.855873+0000","last_became_peered":"2024-03-29T04:07:29.855873+0000","last_unstale":"2024-03-29T04:08:32.803857+0000","last_undegraded":"2024-03-29T04:08:32.803857+0000","last_fullsized":"2024-03-29T04:08:32.803857+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_clean_scrub_stamp":"2024-03-29T04:07:28.661516+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:25:15.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73416,"kb_used_data":46400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94298424,"statfs":{"total":96636764160,"available":96561586176,"internally_reserved":0,"allocated":47513600,"data_stored":47237297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52092,"kb_used_data":25072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94319748,"statfs":{"total":96636764160,"available":96583421952,"internally_reserved":0,"allocated":25673728,"data_stored":25418645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89084,"kb_used_data":62068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94282756,"statfs":{"total":96636764160,"available":96545542144,"internally_reserved":0,"allocated":63557632,"data_stored":63282559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85744,"kb_used_data":58724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94286096,"statfs":{"total":96636764160,"available":96548962304,"internally_reserved":0,"allocated":60133376,"data_stored":59855078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65532,"kb_used_data":38516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94306308,"statfs":{"total":96636764160,"available":96569659392,"internally_reserved":0,"allocated":39440384,"data_stored":39145402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52304,"kb_used_data":25288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94319536,"statfs":{"total":96636764160,"available":96583204864,"internally_reserved":0,"allocated":25894912,"data_stored":25632306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82300,"kb_used_data":55284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94289540,"statfs":{"total":96636764160,"available":96552488960,"internally_reserved":0,"allocated":56610816,"data_stored":56319972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61032,"kb_used_data":34016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94310808,"statfs":{"total":96636764160,"available":96574267392,"internally_reserved":0,"allocated":34832384,"data_stored":34565349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:09:38.435 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:38.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.556+0000 7f39100b1640 1 Processor -- start 2024-03-29T04:09:38.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.557+0000 7f39100b1640 1 -- start start 2024-03-29T04:09:38.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.557+0000 7f39100b1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f390815b370 0x7f39080b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.557+0000 7f39100b1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3908073290 0x7f390815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.557+0000 7f39100b1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3908071ae0 con 0x7f390806f3f0 2024-03-29T04:09:38.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.557+0000 7f39100b1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f3908071dc0 con 0x7f3908073290 2024-03-29T04:09:38.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.557+0000 7f39100b1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f39080720a0 con 0x7f390815b370 2024-03-29T04:09:38.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.558+0000 7f390e64e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3908073290 0x7f390815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.558+0000 7f390e64e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3908073290 0x7f390815f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36268/0 (socket says 172.21.15.97:36268) 2024-03-29T04:09:38.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.558+0000 7f390e64e640 1 -- 172.21.15.97:0/339320720 learned_addr learned my addr 172.21.15.97:0/339320720 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:38.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.558+0000 7f390de4d640 1 --2- 172.21.15.97:0/339320720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f390815b370 0x7f39080b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.558+0000 7f390e64e640 1 -- 172.21.15.97:0/339320720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f390815b370 msgr2=0x7f39080b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:38.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.558+0000 7f390e64e640 1 --2- 172.21.15.97:0/339320720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f390815b370 0x7f39080b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.558+0000 7f390e64e640 1 -- 172.21.15.97:0/339320720 >> v1:172.21.15.154:6789/0 conn(0x7f390806f3f0 legacy=0x7f3908072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.558+0000 7f390e64e640 1 -- 172.21.15.97:0/339320720 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3908072380 con 0x7f3908073290 2024-03-29T04:09:38.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.558+0000 7f390de4d640 1 --2- 172.21.15.97:0/339320720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f390815b370 0x7f39080b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:38.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.559+0000 7f390e64e640 1 --2- 172.21.15.97:0/339320720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3908073290 0x7f390815f0a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3900009e50 tx=0x7f390002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891811adf2037e77 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.559+0000 7f390ce4b640 1 -- 172.21.15.97:0/339320720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390002cb00 con 0x7f3908073290 2024-03-29T04:09:38.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.559+0000 7f390ce4b640 1 -- 172.21.15.97:0/339320720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f390002cca0 con 0x7f3908073290 2024-03-29T04:09:38.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.559+0000 7f390ce4b640 1 -- 172.21.15.97:0/339320720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900034610 con 0x7f3908073290 2024-03-29T04:09:38.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.559+0000 7f39100b1640 1 -- 172.21.15.97:0/339320720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3908073290 msgr2=0x7f390815f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.559+0000 7f39100b1640 1 --2- 172.21.15.97:0/339320720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3908073290 0x7f390815f0a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3900009e50 tx=0x7f390002bf60 comp rx=0 tx=0).stop 2024-03-29T04:09:38.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.559+0000 7f39100b1640 1 -- 172.21.15.97:0/339320720 shutdown_connections 2024-03-29T04:09:38.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.559+0000 7f39100b1640 1 --2- 172.21.15.97:0/339320720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3908073290 0x7f390815f0a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.559+0000 7f39100b1640 1 --2- 172.21.15.97:0/339320720 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f390815b370 0x7f39080b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.559+0000 7f39100b1640 1 -- 172.21.15.97:0/339320720 >> 172.21.15.97:0/339320720 conn(0x7f390806b510 msgr2=0x7f390806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:38.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.559+0000 7f39100b1640 1 -- 172.21.15.97:0/339320720 shutdown_connections 2024-03-29T04:09:38.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.559+0000 7f39100b1640 1 -- 172.21.15.97:0/339320720 wait complete. 2024-03-29T04:09:38.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.561+0000 7f39100b1640 1 Processor -- start 2024-03-29T04:09:38.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.561+0000 7f39100b1640 1 -- start start 2024-03-29T04:09:38.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.561+0000 7f39100b1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f390806f3f0 0x7f3908146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.561+0000 7f39100b1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3908073290 0x7f39081598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.561+0000 7f39100b1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390815b370 0x7f3908159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.561+0000 7f39100b1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39080b3980 con 0x7f390815b370 2024-03-29T04:09:38.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.561+0000 7f39100b1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f39080b3800 con 0x7f390806f3f0 2024-03-29T04:09:38.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.561+0000 7f39100b1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f390815b750 con 0x7f3908073290 2024-03-29T04:09:38.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.561+0000 7f390e64e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390815b370 0x7f3908159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.561+0000 7f390e64e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390815b370 0x7f3908159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36280/0 (socket says 172.21.15.97:36280) 2024-03-29T04:09:38.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.561+0000 7f390e64e640 1 -- 172.21.15.97:0/3117802688 learned_addr learned my addr 172.21.15.97:0/3117802688 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:38.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f390de4d640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f390806f3f0 0x7f3908146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f390d64c640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3908073290 0x7f39081598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f390e64e640 1 -- 172.21.15.97:0/3117802688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3908073290 msgr2=0x7f39081598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f390e64e640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3908073290 0x7f39081598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f390e64e640 1 -- 172.21.15.97:0/3117802688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f390806f3f0 msgr2=0x7f3908146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f390e64e640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f390806f3f0 0x7f3908146690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f390e64e640 1 -- 172.21.15.97:0/3117802688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3908157ac0 con 0x7f390815b370 2024-03-29T04:09:38.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f390de4d640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f390806f3f0 0x7f3908146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:38.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f390d64c640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3908073290 0x7f39081598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:38.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f390e64e640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390815b370 0x7f3908159e20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3900009f70 tx=0x7f3900004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f38fe7fc640 1 -- 172.21.15.97:0/3117802688 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900043020 con 0x7f390815b370 2024-03-29T04:09:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f38fe7fc640 1 -- 172.21.15.97:0/3117802688 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f390002ce50 con 0x7f390815b370 2024-03-29T04:09:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f38fe7fc640 1 -- 172.21.15.97:0/3117802688 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900009510 con 0x7f390815b370 2024-03-29T04:09:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f39100b1640 1 -- 172.21.15.97:0/3117802688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3908157da0 con 0x7f390815b370 2024-03-29T04:09:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.562+0000 7f39100b1640 1 -- 172.21.15.97:0/3117802688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3908146bd0 con 0x7f390815b370 2024-03-29T04:09:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.563+0000 7f39100b1640 1 -- 172.21.15.97:0/3117802688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3908068640 con 0x7f390815b370 2024-03-29T04:09:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.563+0000 7f38fe7fc640 1 -- 172.21.15.97:0/3117802688 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3900046020 con 0x7f390815b370 2024-03-29T04:09:38.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.564+0000 7f38fe7fc640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f38ec06c530 0x7f38ec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.564+0000 7f38fe7fc640 1 -- 172.21.15.97:0/3117802688 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f39000ada70 con 0x7f390815b370 2024-03-29T04:09:38.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.564+0000 7f390de4d640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f38ec06c530 0x7f38ec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.564+0000 7f390de4d640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f38ec06c530 0x7f38ec06e9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f38f8006fd0 tx=0x7f38f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.568+0000 7f38fe7fc640 1 -- 172.21.15.97:0/3117802688 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f390004d030 con 0x7f390815b370 2024-03-29T04:09:38.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.745+0000 7f39100b1640 1 -- 172.21.15.97:0/3117802688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f390806d9d0 con 0x7f390815b370 2024-03-29T04:09:38.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.746+0000 7f38fe7fc640 1 -- 172.21.15.97:0/3117802688 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13020 (secure 0 0 0) 0x7f390007c390 con 0x7f390815b370 2024-03-29T04:09:38.725 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:38.726 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":35,"fsid":"51f8bdaf-3a8d-4eb9-b08c-b6e8e46b575f","created":"2024-03-29T04:07:16.444011+0000","modified":"2024-03-29T04:09:35.358970+0000","last_up_change":"2024-03-29T04:07:27.623588+0000","last_in_change":"2024-03-29T04:07:18.654511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:07:28.653487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c6327747-649d-4d0b-9cbe-78f197b0a784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6801","nonce":2006100648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6803","nonce":2006100648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6807","nonce":2006100648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2006100648},{"type":"v1","addr":"172.21.15.97:6805","nonce":2006100648}]},"public_addr":"172.21.15.97:6801/2006100648","cluster_addr":"172.21.15.97:6803/2006100648","heartbeat_back_addr":"172.21.15.97:6807/2006100648","heartbeat_front_addr":"172.21.15.97:6805/2006100648","state":["exists","up"]},{"osd":1,"uuid":"176691c0-02fd-40e8-abb2-115fce34d4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6809","nonce":4164988844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6811","nonce":4164988844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6815","nonce":4164988844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4164988844},{"type":"v1","addr":"172.21.15.97:6813","nonce":4164988844}]},"public_addr":"172.21.15.97:6809/4164988844","cluster_addr":"172.21.15.97:6811/4164988844","heartbeat_back_addr":"172.21.15.97:6815/4164988844","heartbeat_front_addr":"172.21.15.97:6813/4164988844","state":["exists","up"]},{"osd":2,"uuid":"306572ef-01fe-44d9-8488-61fc93883d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6817","nonce":127556154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6819","nonce":127556154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6823","nonce":127556154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":127556154},{"type":"v1","addr":"172.21.15.97:6821","nonce":127556154}]},"public_addr":"172.21.15.97:6817/127556154","cluster_addr":"172.21.15.97:6819/127556154","heartbeat_back_addr":"172.21.15.97:6823/127556154","heartbeat_front_addr":"172.21.15.97:6821/127556154","state":["exists","up"]},{"osd":3,"uuid":"59b4d556-3332-4c8e-b6f7-231ed9d64f7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6825","nonce":1504398439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6827","nonce":1504398439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6831","nonce":1504398439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1504398439},{"type":"v1","addr":"172.21.15.97:6829","nonce":1504398439}]},"public_addr":"172.21.15.97:6825/1504398439","cluster_addr":"172.21.15.97:6827/1504398439","heartbeat_back_addr":"172.21.15.97:6831/1504398439","heartbeat_front_addr":"172.21.15.97:6829/1504398439","state":["exists","up"]},{"osd":4,"uuid":"1910d100-2ee1-4a4c-8eb5-427373f8d0c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6825","nonce":2981976178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6827","nonce":2981976178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6831","nonce":2981976178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2981976178},{"type":"v1","addr":"172.21.15.154:6829","nonce":2981976178}]},"public_addr":"172.21.15.154:6825/2981976178","cluster_addr":"172.21.15.154:6827/2981976178","heartbeat_back_addr":"172.21.15.154:6831/2981976178","heartbeat_front_addr":"172.21.15.154:6829/2981976178","state":["exists","up"]},{"osd":5,"uuid":"a15d2654-a296-4f18-8db2-e3de9ff567f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6809","nonce":2773817748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6811","nonce":2773817748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6815","nonce":2773817748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2773817748},{"type":"v1","addr":"172.21.15.154:6813","nonce":2773817748}]},"public_addr":"172.21.15.154:6809/2773817748","cluster_addr":"172.21.15.154:6811/2773817748","heartbeat_back_addr":"172.21.15.154:6815/2773817748","heartbeat_front_addr":"172.21.15.154:6813/2773817748","state":["exists","up"]},{"osd":6,"uuid":"01c89cd8-ac08-4e7d-90f6-0ef9e1c0f4ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6801","nonce":1169845562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6803","nonce":1169845562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6807","nonce":1169845562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1169845562},{"type":"v1","addr":"172.21.15.154:6805","nonce":1169845562}]},"public_addr":"172.21.15.154:6801/1169845562","cluster_addr":"172.21.15.154:6803/1169845562","heartbeat_back_addr":"172.21.15.154:6807/1169845562","heartbeat_front_addr":"172.21.15.154:6805/1169845562","state":["exists","up"]},{"osd":7,"uuid":"a9236f6d-34f1-4ace-a310-b63fad918bc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6817","nonce":2375296016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6819","nonce":2375296016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6823","nonce":2375296016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2375296016},{"type":"v1","addr":"172.21.15.154:6821","nonce":2375296016}]},"public_addr":"172.21.15.154:6817/2375296016","cluster_addr":"172.21.15.154:6819/2375296016","heartbeat_back_addr":"172.21.15.154:6823/2375296016","heartbeat_front_addr":"172.21.15.154:6821/2375296016","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.816224+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.810301+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.800181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.878486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.868674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.893773+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.752941+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:07:23.867065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/3142898446":"2024-03-30T04:09:32.088630+0000","172.21.15.97:6834/3142898446":"2024-03-30T04:09:32.088630+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:38.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.748+0000 7f39100b1640 1 -- 172.21.15.97:0/3117802688 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f38ec06c530 msgr2=0x7f38ec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.748+0000 7f39100b1640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f38ec06c530 0x7f38ec06e9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f38f8006fd0 tx=0x7f38f8008040 comp rx=0 tx=0).stop 2024-03-29T04:09:38.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.748+0000 7f39100b1640 1 -- 172.21.15.97:0/3117802688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390815b370 msgr2=0x7f3908159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.748+0000 7f39100b1640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390815b370 0x7f3908159e20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3900009f70 tx=0x7f3900004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:38.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.748+0000 7f39100b1640 1 -- 172.21.15.97:0/3117802688 shutdown_connections 2024-03-29T04:09:38.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.748+0000 7f39100b1640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f38ec06c530 0x7f38ec06e9f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.748+0000 7f39100b1640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f390815b370 0x7f3908159e20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.748+0000 7f39100b1640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3908073290 0x7f39081598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.748+0000 7f39100b1640 1 --2- 172.21.15.97:0/3117802688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f390806f3f0 0x7f3908146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.748+0000 7f39100b1640 1 -- 172.21.15.97:0/3117802688 >> 172.21.15.97:0/3117802688 conn(0x7f390806b510 msgr2=0x7f390806e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:38.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.748+0000 7f39100b1640 1 -- 172.21.15.97:0/3117802688 shutdown_connections 2024-03-29T04:09:38.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:38.748+0000 7f39100b1640 1 -- 172.21.15.97:0/3117802688 wait complete. 2024-03-29T04:09:39.739 INFO:tasks.ceph:Scrubbing osd.0 2024-03-29T04:09:39.739 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:09:39.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.875+0000 7f811444a640 1 Processor -- start 2024-03-29T04:09:39.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.875+0000 7f811444a640 1 -- start start 2024-03-29T04:09:39.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.876+0000 7f811444a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f810c15e780 0x7f810c162ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.876+0000 7f811444a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f810c0d4f20 0x7f810c0b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.876+0000 7f811444a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f810c15eb60 0x7f810c0d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.876+0000 7f811444a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f810c163430 con 0x7f810c0d4f20 2024-03-29T04:09:39.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.876+0000 7f811444a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f810c0cfde0 con 0x7f810c15eb60 2024-03-29T04:09:39.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.876+0000 7f811444a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f810c0b37c0 con 0x7f810c15e780 2024-03-29T04:09:39.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.876+0000 7f81121e6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f810c15e780 0x7f810c162ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.876+0000 7f81119e5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f810c0d4f20 0x7f810c0b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.876+0000 7f81119e5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f810c0d4f20 0x7f810c0b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36290/0 (socket says 172.21.15.97:36290) 2024-03-29T04:09:39.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.876+0000 7f81119e5640 1 -- 172.21.15.97:0/3191840213 learned_addr learned my addr 172.21.15.97:0/3191840213 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:39.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.877+0000 7f81129e7640 1 --2- 172.21.15.97:0/3191840213 >> v2:172.21.15.154:3300/0 conn(0x7f810c15eb60 0x7f810c0d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.877+0000 7f81119e5640 1 -- 172.21.15.97:0/3191840213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f810c15e780 msgr2=0x7f810c162ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:39.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.877+0000 7f81119e5640 1 --2- 172.21.15.97:0/3191840213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f810c15e780 0x7f810c162ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.877+0000 7f81119e5640 1 -- 172.21.15.97:0/3191840213 >> v2:172.21.15.154:3300/0 conn(0x7f810c15eb60 msgr2=0x7f810c0d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:39.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.877+0000 7f81119e5640 1 --2- 172.21.15.97:0/3191840213 >> v2:172.21.15.154:3300/0 conn(0x7f810c15eb60 0x7f810c0d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.877+0000 7f81119e5640 1 -- 172.21.15.97:0/3191840213 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c0d53e0 con 0x7f810c0d4f20 2024-03-29T04:09:39.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.877+0000 7f81119e5640 1 --2- 172.21.15.97:0/3191840213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f810c0d4f20 0x7f810c0b1170 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f810c08e2e0 tx=0x7f80fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40dfdcc54ee1a8f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.878+0000 7f81111e4640 1 -- 172.21.15.97:0/3191840213 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc035020 con 0x7f810c0d4f20 2024-03-29T04:09:39.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.878+0000 7f81111e4640 1 -- 172.21.15.97:0/3191840213 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80fc031640 con 0x7f810c0d4f20 2024-03-29T04:09:39.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.878+0000 7f81111e4640 1 -- 172.21.15.97:0/3191840213 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc02d040 con 0x7f810c0d4f20 2024-03-29T04:09:39.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.878+0000 7f811444a640 1 -- 172.21.15.97:0/3191840213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f810c0d4f20 msgr2=0x7f810c0b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.878+0000 7f811444a640 1 --2- 172.21.15.97:0/3191840213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f810c0d4f20 0x7f810c0b1170 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f810c08e2e0 tx=0x7f80fc02b8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.878+0000 7f811444a640 1 -- 172.21.15.97:0/3191840213 shutdown_connections 2024-03-29T04:09:39.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.878+0000 7f811444a640 1 --2- 172.21.15.97:0/3191840213 >> v2:172.21.15.154:3300/0 conn(0x7f810c15eb60 0x7f810c0d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.878+0000 7f811444a640 1 --2- 172.21.15.97:0/3191840213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f810c0d4f20 0x7f810c0b1170 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.878+0000 7f811444a640 1 --2- 172.21.15.97:0/3191840213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f810c15e780 0x7f810c162ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.878+0000 7f811444a640 1 -- 172.21.15.97:0/3191840213 >> 172.21.15.97:0/3191840213 conn(0x7f810c08bfc0 msgr2=0x7f810c0d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.878+0000 7f811444a640 1 -- 172.21.15.97:0/3191840213 shutdown_connections 2024-03-29T04:09:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.878+0000 7f811444a640 1 -- 172.21.15.97:0/3191840213 wait complete. 2024-03-29T04:09:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.880+0000 7f811444a640 1 Processor -- start 2024-03-29T04:09:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.880+0000 7f811444a640 1 -- start start 2024-03-29T04:09:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.880+0000 7f811444a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f810c0d4f20 0x7f810c0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.880+0000 7f811444a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f810c15e780 0x7f810c0b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.880+0000 7f811444a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f810c15eb60 0x7f810c0b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.880+0000 7f811444a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f810c0cf440 con 0x7f810c0d4f20 2024-03-29T04:09:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.880+0000 7f811444a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f810c0cfa70 con 0x7f810c15eb60 2024-03-29T04:09:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.880+0000 7f811444a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f810c0b16b0 con 0x7f810c15e780 2024-03-29T04:09:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.880+0000 7f81121e6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f810c0d4f20 0x7f810c0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f81121e6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f810c0d4f20 0x7f810c0c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36306/0 (socket says 172.21.15.97:36306) 2024-03-29T04:09:39.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f81121e6640 1 -- 172.21.15.97:0/2635542136 learned_addr learned my addr 172.21.15.97:0/2635542136 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:39.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f81119e5640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f810c15e780 0x7f810c0b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f81129e7640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f810c15eb60 0x7f810c0b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f81121e6640 1 -- 172.21.15.97:0/2635542136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f810c15e780 msgr2=0x7f810c0b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f81121e6640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f810c15e780 0x7f810c0b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f81121e6640 1 -- 172.21.15.97:0/2635542136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f810c15eb60 msgr2=0x7f810c0b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f81121e6640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f810c15eb60 0x7f810c0b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f81121e6640 1 -- 172.21.15.97:0/2635542136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c1f51d0 con 0x7f810c0d4f20 2024-03-29T04:09:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f81121e6640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f810c0d4f20 0x7f810c0c9bf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f810800e9b0 tx=0x7f810800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f8102ffd640 1 -- 172.21.15.97:0/2635542136 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108017070 con 0x7f810c0d4f20 2024-03-29T04:09:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f8102ffd640 1 -- 172.21.15.97:0/2635542136 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810800c9a0 con 0x7f810c0d4f20 2024-03-29T04:09:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f811444a640 1 -- 172.21.15.97:0/2635542136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810c15f1f0 con 0x7f810c0d4f20 2024-03-29T04:09:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f8102ffd640 1 -- 172.21.15.97:0/2635542136 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810800cd90 con 0x7f810c0d4f20 2024-03-29T04:09:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.881+0000 7f811444a640 1 -- 172.21.15.97:0/2635542136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810c15f7b0 con 0x7f810c0d4f20 2024-03-29T04:09:39.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.882+0000 7f811444a640 1 -- 172.21.15.97:0/2635542136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f810c15efe0 con 0x7f810c0d4f20 2024-03-29T04:09:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.883+0000 7f8102ffd640 1 -- 172.21.15.97:0/2635542136 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f810801a070 con 0x7f810c0d4f20 2024-03-29T04:09:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.883+0000 7f8102ffd640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f80f006c530 0x7f80f006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.883+0000 7f8102ffd640 1 -- 172.21.15.97:0/2635542136 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f810808b340 con 0x7f810c0d4f20 2024-03-29T04:09:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.883+0000 7f81119e5640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f80f006c530 0x7f80f006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.883+0000 7f8102ffd640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] conn(0x7f80f00723f0 0x7f80f0074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.883+0000 7f8102ffd640 1 -- 172.21.15.97:0/2635542136 --> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80f0074ec0 con 0x7f80f00723f0 2024-03-29T04:09:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.883+0000 7f8102ffd640 1 -- 172.21.15.97:0/2635542136 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8108092050 con 0x7f810c0d4f20 2024-03-29T04:09:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.883+0000 7f81129e7640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] conn(0x7f80f00723f0 0x7f80f0074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.883+0000 7f81119e5640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f80f006c530 0x7f80f006e9f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f80fc036000 tx=0x7f80fc03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.884+0000 7f81129e7640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] conn(0x7f80f00723f0 0x7f80f0074810 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.885+0000 7f8102ffd640 1 -- 172.21.15.97:0/2635542136 <== osd.0 v2:172.21.15.97:6800/2006100648 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80f0074ec0 con 0x7f80f00723f0 2024-03-29T04:09:39.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.898+0000 7f811444a640 1 -- 172.21.15.97:0/2635542136 --> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f810c068640 con 0x7f80f00723f0 2024-03-29T04:09:39.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.898+0000 7f8102ffd640 1 -- 172.21.15.97:0/2635542136 <== osd.0 v2:172.21.15.97:6800/2006100648 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f810c068640 con 0x7f80f00723f0 2024-03-29T04:09:39.878 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:09:39.878 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:09:39.879 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:09:39.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.899+0000 7f811444a640 1 -- 172.21.15.97:0/2635542136 >> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] conn(0x7f80f00723f0 msgr2=0x7f80f0074810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.899+0000 7f811444a640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.97:6800/2006100648,v1:172.21.15.97:6801/2006100648] conn(0x7f80f00723f0 0x7f80f0074810 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.899+0000 7f811444a640 1 -- 172.21.15.97:0/2635542136 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f80f006c530 msgr2=0x7f80f006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.899+0000 7f811444a640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f80f006c530 0x7f80f006e9f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f80fc036000 tx=0x7f80fc03c040 comp rx=0 tx=0).stop 2024-03-29T04:09:39.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.899+0000 7f811444a640 1 -- 172.21.15.97:0/2635542136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f810c0d4f20 msgr2=0x7f810c0c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.899+0000 7f811444a640 1 --2- 172.21.15.97:0/2635542136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f810c0d4f20 0x7f810c0c9bf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f810800e9b0 tx=0x7f810800ee70 comp rx=0 tx=0).stop 2024-03-29T04:09:39.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.899+0000 7f81129e7640 1 -- 172.21.15.97:0/2635542136 reap_dead start 2024-03-29T04:09:39.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.899+0000 7f811444a640 1 -- 172.21.15.97:0/2635542136 shutdown_connections 2024-03-29T04:09:39.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.899+0000 7f811444a640 1 -- 172.21.15.97:0/2635542136 >> 172.21.15.97:0/2635542136 conn(0x7f810c08bfc0 msgr2=0x7f810c06d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:39.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.899+0000 7f811444a640 1 -- 172.21.15.97:0/2635542136 shutdown_connections 2024-03-29T04:09:39.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:39.899+0000 7f811444a640 1 -- 172.21.15.97:0/2635542136 wait complete. 2024-03-29T04:09:39.889 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-29T04:09:39.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.012+0000 7f84e74b3640 1 Processor -- start 2024-03-29T04:09:39.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.012+0000 7f84e74b3640 1 -- start start 2024-03-29T04:09:39.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.013+0000 7f84e74b3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f84e00ce800 0x7f84e00c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.013+0000 7f84e74b3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e00c7f90 con 0x7f84e006c000 2024-03-29T04:09:39.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.013+0000 7f84e74b3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e00cdd20 con 0x7f84e006b510 2024-03-29T04:09:39.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.013+0000 7f84e74b3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f84e015ece0 con 0x7f84e00ce800 2024-03-29T04:09:39.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.013+0000 7f84e4a4e640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f84e006c000 0x7f84e00cee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57402/0 (socket says 172.21.15.97:57402) 2024-03-29T04:09:39.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.013+0000 7f84e4a4e640 1 -- 172.21.15.97:0/3588863146 learned_addr learned my addr 172.21.15.97:0/3588863146 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:39.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.013+0000 7f84e5a50640 1 --2- 172.21.15.97:0/3588863146 >> v2:172.21.15.154:3300/0 conn(0x7f84e00ce800 0x7f84e00c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.014+0000 7f84d7fff640 1 -- 172.21.15.97:0/3588863146 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2946703182 0 0) 0x7f84e00c7f90 con 0x7f84e006c000 2024-03-29T04:09:39.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.014+0000 7f84d7fff640 1 -- 172.21.15.97:0/3588863146 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84cc003520 con 0x7f84e006c000 2024-03-29T04:09:39.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.014+0000 7f84d7fff640 1 -- 172.21.15.97:0/3588863146 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 935874347 0 0) 0x7f84e00cdd20 con 0x7f84e006b510 2024-03-29T04:09:39.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.014+0000 7f84d7fff640 1 -- 172.21.15.97:0/3588863146 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84e00c7f90 con 0x7f84e006b510 2024-03-29T04:09:39.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.014+0000 7f84e5a50640 1 -- 172.21.15.97:0/3588863146 >> v1:172.21.15.154:6789/0 conn(0x7f84e006b510 legacy=0x7f84e006b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.014+0000 7f84e5a50640 1 -- 172.21.15.97:0/3588863146 >> v1:172.21.15.97:6789/0 conn(0x7f84e006c000 legacy=0x7f84e00cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.014+0000 7f84e5a50640 1 -- 172.21.15.97:0/3588863146 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e00ce000 con 0x7f84e00ce800 2024-03-29T04:09:39.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.014+0000 7f84e5a50640 1 --2- 172.21.15.97:0/3588863146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e00ce800 0x7f84e00c7840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009d90 tx=0x7f84d802be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dc15548488889b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.015+0000 7f84d7fff640 1 -- 172.21.15.97:0/3588863146 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8039070 con 0x7f84e00ce800 2024-03-29T04:09:39.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.015+0000 7f84d7fff640 1 -- 172.21.15.97:0/3588863146 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d802cbd0 con 0x7f84e00ce800 2024-03-29T04:09:39.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.015+0000 7f84d7fff640 1 -- 172.21.15.97:0/3588863146 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8034440 con 0x7f84e00ce800 2024-03-29T04:09:39.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.015+0000 7f84e74b3640 1 -- 172.21.15.97:0/3588863146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e00ce800 msgr2=0x7f84e00c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.015+0000 7f84e74b3640 1 --2- 172.21.15.97:0/3588863146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e00ce800 0x7f84e00c7840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009d90 tx=0x7f84d802be20 comp rx=0 tx=0).stop 2024-03-29T04:09:39.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.015+0000 7f84e74b3640 1 -- 172.21.15.97:0/3588863146 shutdown_connections 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.015+0000 7f84e74b3640 1 --2- 172.21.15.97:0/3588863146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e00ce800 0x7f84e00c7840 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.015+0000 7f84e74b3640 1 -- 172.21.15.97:0/3588863146 >> 172.21.15.97:0/3588863146 conn(0x7f84e00b2e60 msgr2=0x7f84e00b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.015+0000 7f84e74b3640 1 -- 172.21.15.97:0/3588863146 shutdown_connections 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.015+0000 7f84e74b3640 1 -- 172.21.15.97:0/3588863146 wait complete. 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.017+0000 7f84e74b3640 1 Processor -- start 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.017+0000 7f84e74b3640 1 -- start start 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.017+0000 7f84e74b3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84e006b510 0x7f84e014d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.017+0000 7f84e74b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e006c000 0x7f84e00b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.017+0000 7f84e74b3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e00ce800 0x7f84e00cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.017+0000 7f84e74b3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84e00ce370 con 0x7f84e00ce800 2024-03-29T04:09:40.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.017+0000 7f84e74b3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f84e00cdad0 con 0x7f84e006c000 2024-03-29T04:09:40.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.017+0000 7f84e74b3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f84e00b1c80 con 0x7f84e006b510 2024-03-29T04:09:40.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.017+0000 7f84e5a50640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e00ce800 0x7f84e00cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.017+0000 7f84e5a50640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e00ce800 0x7f84e00cff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36318/0 (socket says 172.21.15.97:36318) 2024-03-29T04:09:40.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.017+0000 7f84e5a50640 1 -- 172.21.15.97:0/3607658092 learned_addr learned my addr 172.21.15.97:0/3607658092 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:40.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.017+0000 7f84e4a4e640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e006c000 0x7f84e00b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.017+0000 7f84e524f640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84e006b510 0x7f84e014d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.018+0000 7f84e5a50640 1 -- 172.21.15.97:0/3607658092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84e006b510 msgr2=0x7f84e014d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.018+0000 7f84e5a50640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84e006b510 0x7f84e014d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.018+0000 7f84e5a50640 1 -- 172.21.15.97:0/3607658092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e006c000 msgr2=0x7f84e00b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.018+0000 7f84e5a50640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e006c000 0x7f84e00b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.018+0000 7f84e5a50640 1 -- 172.21.15.97:0/3607658092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e014d610 con 0x7f84e00ce800 2024-03-29T04:09:40.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.018+0000 7f84e5a50640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e00ce800 0x7f84e00cff00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009d90 tx=0x7f84d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.018+0000 7f84d5ffb640 1 -- 172.21.15.97:0/3607658092 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8039050 con 0x7f84e00ce800 2024-03-29T04:09:40.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.018+0000 7f84d5ffb640 1 -- 172.21.15.97:0/3607658092 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d8045070 con 0x7f84e00ce800 2024-03-29T04:09:40.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.018+0000 7f84e74b3640 1 -- 172.21.15.97:0/3607658092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e00b5d00 con 0x7f84e00ce800 2024-03-29T04:09:40.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.018+0000 7f84d5ffb640 1 -- 172.21.15.97:0/3607658092 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8009450 con 0x7f84e00ce800 2024-03-29T04:09:40.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.018+0000 7f84e74b3640 1 -- 172.21.15.97:0/3607658092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e00b62c0 con 0x7f84e00ce800 2024-03-29T04:09:40.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.019+0000 7f84e74b3640 1 -- 172.21.15.97:0/3607658092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e0068d50 con 0x7f84e00ce800 2024-03-29T04:09:40.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.020+0000 7f84d5ffb640 1 -- 172.21.15.97:0/3607658092 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f84d803e850 con 0x7f84e00ce800 2024-03-29T04:09:40.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.020+0000 7f84d5ffb640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f84c406c530 0x7f84c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.020+0000 7f84e524f640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f84c406c530 0x7f84c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.020+0000 7f84d5ffb640 1 -- 172.21.15.97:0/3607658092 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f84d80ae0d0 con 0x7f84e00ce800 2024-03-29T04:09:40.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.021+0000 7f84e524f640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f84c406c530 0x7f84c406e9f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f84d0007fd0 tx=0x7f84d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.024+0000 7f84d5ffb640 1 -- 172.21.15.97:0/3607658092 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f84d807c550 con 0x7f84e00ce800 2024-03-29T04:09:40.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.207+0000 7f84e74b3640 1 -- 172.21.15.97:0/3607658092 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f84e00cf460 con 0x7f84c406c530 2024-03-29T04:09:40.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.207+0000 7f84d5ffb640 1 -- 172.21.15.97:0/3607658092 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f84e00cf460 con 0x7f84c406c530 2024-03-29T04:09:40.187 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 0 to deep-scrub 2024-03-29T04:09:40.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.209+0000 7f84e74b3640 1 -- 172.21.15.97:0/3607658092 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f84c406c530 msgr2=0x7f84c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.209+0000 7f84e74b3640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f84c406c530 0x7f84c406e9f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f84d0007fd0 tx=0x7f84d0009040 comp rx=0 tx=0).stop 2024-03-29T04:09:40.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.209+0000 7f84e74b3640 1 -- 172.21.15.97:0/3607658092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e00ce800 msgr2=0x7f84e00cff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.209+0000 7f84e74b3640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e00ce800 0x7f84e00cff00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009d90 tx=0x7f84d8004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:40.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.209+0000 7f84e74b3640 1 -- 172.21.15.97:0/3607658092 shutdown_connections 2024-03-29T04:09:40.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.209+0000 7f84e74b3640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f84c406c530 0x7f84c406e9f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.209+0000 7f84e74b3640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84e00ce800 0x7f84e00cff00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.209+0000 7f84e74b3640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e006c000 0x7f84e00b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.209+0000 7f84e74b3640 1 --2- 172.21.15.97:0/3607658092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84e006b510 0x7f84e014d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.209+0000 7f84e74b3640 1 -- 172.21.15.97:0/3607658092 >> 172.21.15.97:0/3607658092 conn(0x7f84e00b2e60 msgr2=0x7f84e006fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.210+0000 7f84e74b3640 1 -- 172.21.15.97:0/3607658092 shutdown_connections 2024-03-29T04:09:40.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.210+0000 7f84e74b3640 1 -- 172.21.15.97:0/3607658092 wait complete. 2024-03-29T04:09:40.203 INFO:tasks.ceph:Scrubbing osd.1 2024-03-29T04:09:40.203 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:09:40.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.329+0000 7fba6e4c1640 1 Processor -- start 2024-03-29T04:09:40.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.330+0000 7fba6e4c1640 1 -- start start 2024-03-29T04:09:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.330+0000 7fba6e4c1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba6815ab00 0x7fba680b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.330+0000 7fba6e4c1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fba68072870 0x7fba6815ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.330+0000 7fba6e4c1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba680ad660 con 0x7fba6815e7b0 2024-03-29T04:09:40.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.330+0000 7fba6e4c1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fba6815f150 con 0x7fba68072870 2024-03-29T04:09:40.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.330+0000 7fba6e4c1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fba6806d860 con 0x7fba6815ab00 2024-03-29T04:09:40.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.331+0000 7fba67fff640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba6815ab00 0x7fba680b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.331+0000 7fba6ca5e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fba68072870 0x7fba6815ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.331+0000 7fba677fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fba6815e7b0 0x7fba6806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:40352/0 (socket says 172.21.15.97:40352) 2024-03-29T04:09:40.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.331+0000 7fba677fe640 1 -- 172.21.15.97:0/3714646798 learned_addr learned my addr 172.21.15.97:0/3714646798 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:40.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.331+0000 7fba67fff640 1 -- 172.21.15.97:0/3714646798 >> v2:172.21.15.154:3300/0 conn(0x7fba68072870 msgr2=0x7fba6815ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:40.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.331+0000 7fba67fff640 1 --2- 172.21.15.97:0/3714646798 >> v2:172.21.15.154:3300/0 conn(0x7fba68072870 0x7fba6815ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.331+0000 7fba67fff640 1 -- 172.21.15.97:0/3714646798 >> v1:172.21.15.154:6789/0 conn(0x7fba6815e7b0 legacy=0x7fba6806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.331+0000 7fba67fff640 1 -- 172.21.15.97:0/3714646798 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba680b4ab0 con 0x7fba6815ab00 2024-03-29T04:09:40.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.331+0000 7fba6ca5e640 1 --2- 172.21.15.97:0/3714646798 >> v2:172.21.15.154:3300/0 conn(0x7fba68072870 0x7fba6815ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:40.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.332+0000 7fba67fff640 1 --2- 172.21.15.97:0/3714646798 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba6815ab00 0x7fba680b43f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0097a0 tx=0x7fba5c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=231350da05f5944b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.332+0000 7fba66ffd640 1 -- 172.21.15.97:0/3714646798 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0315a0 con 0x7fba6815ab00 2024-03-29T04:09:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.332+0000 7fba66ffd640 1 -- 172.21.15.97:0/3714646798 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba5c037070 con 0x7fba6815ab00 2024-03-29T04:09:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.332+0000 7fba66ffd640 1 -- 172.21.15.97:0/3714646798 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c031960 con 0x7fba6815ab00 2024-03-29T04:09:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.332+0000 7fba6e4c1640 1 -- 172.21.15.97:0/3714646798 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba6815ab00 msgr2=0x7fba680b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.332+0000 7fba6e4c1640 1 --2- 172.21.15.97:0/3714646798 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba6815ab00 0x7fba680b43f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0097a0 tx=0x7fba5c02b840 comp rx=0 tx=0).stop 2024-03-29T04:09:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.332+0000 7fba6e4c1640 1 -- 172.21.15.97:0/3714646798 shutdown_connections 2024-03-29T04:09:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.332+0000 7fba6e4c1640 1 --2- 172.21.15.97:0/3714646798 >> v2:172.21.15.154:3300/0 conn(0x7fba68072870 0x7fba6815ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.332+0000 7fba6e4c1640 1 --2- 172.21.15.97:0/3714646798 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba6815ab00 0x7fba680b43f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.332+0000 7fba6e4c1640 1 -- 172.21.15.97:0/3714646798 >> 172.21.15.97:0/3714646798 conn(0x7fba6806b6f0 msgr2=0x7fba6806bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.333+0000 7fba6e4c1640 1 -- 172.21.15.97:0/3714646798 shutdown_connections 2024-03-29T04:09:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.333+0000 7fba6e4c1640 1 -- 172.21.15.97:0/3714646798 wait complete. 2024-03-29T04:09:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.334+0000 7fba6e4c1640 1 Processor -- start 2024-03-29T04:09:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.334+0000 7fba6e4c1640 1 -- start start 2024-03-29T04:09:40.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6e4c1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba68072870 0x7fba68099670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6e4c1640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba6815ab00 0x7fba680b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6e4c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba6815e7b0 0x7fba681451f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6e4c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba680b5070 con 0x7fba6815e7b0 2024-03-29T04:09:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6e4c1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fba6806db40 con 0x7fba68072870 2024-03-29T04:09:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6e4c1640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fba6815aee0 con 0x7fba6815ab00 2024-03-29T04:09:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba67fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba68072870 0x7fba68099670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba677fe640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba6815ab00 0x7fba680b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6ca5e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba6815e7b0 0x7fba681451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba67fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba68072870 0x7fba68099670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.97:53024/0 (socket says 172.21.15.97:53024) 2024-03-29T04:09:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba67fff640 1 -- 172.21.15.97:0/1161598903 learned_addr learned my addr 172.21.15.97:0/1161598903 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6ca5e640 1 -- 172.21.15.97:0/1161598903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba6815ab00 msgr2=0x7fba680b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6ca5e640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba6815ab00 0x7fba680b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6ca5e640 1 -- 172.21.15.97:0/1161598903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba68072870 msgr2=0x7fba68099670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6ca5e640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba68072870 0x7fba68099670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6ca5e640 1 -- 172.21.15.97:0/1161598903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba680b2fd0 con 0x7fba6815e7b0 2024-03-29T04:09:40.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba67fff640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba68072870 0x7fba68099670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:40.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba677fe640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba6815ab00 0x7fba680b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:40.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6ca5e640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba6815e7b0 0x7fba681451f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fba6000efc0 tx=0x7fba6000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba64ff9640 1 -- 172.21.15.97:0/1161598903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60018030 con 0x7fba6815e7b0 2024-03-29T04:09:40.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba64ff9640 1 -- 172.21.15.97:0/1161598903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6000cd30 con 0x7fba6815e7b0 2024-03-29T04:09:40.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba6e4c1640 1 -- 172.21.15.97:0/1161598903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba68158060 con 0x7fba6815e7b0 2024-03-29T04:09:40.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.335+0000 7fba64ff9640 1 -- 172.21.15.97:0/1161598903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60003bb0 con 0x7fba6815e7b0 2024-03-29T04:09:40.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.336+0000 7fba6e4c1640 1 -- 172.21.15.97:0/1161598903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba68157d90 con 0x7fba6815e7b0 2024-03-29T04:09:40.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.336+0000 7fba6e4c1640 1 -- 172.21.15.97:0/1161598903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba680b11f0 con 0x7fba6815e7b0 2024-03-29T04:09:40.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.337+0000 7fba64ff9640 1 -- 172.21.15.97:0/1161598903 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fba60005d10 con 0x7fba6815e7b0 2024-03-29T04:09:40.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.337+0000 7fba64ff9640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fba5406c560 0x7fba5406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.337+0000 7fba64ff9640 1 -- 172.21.15.97:0/1161598903 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fba6008bc60 con 0x7fba6815e7b0 2024-03-29T04:09:40.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.337+0000 7fba67fff640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fba5406c560 0x7fba5406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.337+0000 7fba64ff9640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] conn(0x7fba54072420 0x7fba54074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.337+0000 7fba64ff9640 1 -- 172.21.15.97:0/1161598903 --> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba54074ef0 con 0x7fba54072420 2024-03-29T04:09:40.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.337+0000 7fba64ff9640 1 -- 172.21.15.97:0/1161598903 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fba60091070 con 0x7fba6815e7b0 2024-03-29T04:09:40.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.338+0000 7fba677fe640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] conn(0x7fba54072420 0x7fba54074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.338+0000 7fba67fff640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fba5406c560 0x7fba5406ea20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0097a0 tx=0x7fba5c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.338+0000 7fba677fe640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] conn(0x7fba54072420 0x7fba54074840 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.338+0000 7fba64ff9640 1 -- 172.21.15.97:0/1161598903 <== osd.1 v2:172.21.15.97:6808/4164988844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fba54074ef0 con 0x7fba54072420 2024-03-29T04:09:40.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.351+0000 7fba6e4c1640 1 -- 172.21.15.97:0/1161598903 --> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fba68158ae0 con 0x7fba54072420 2024-03-29T04:09:40.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.352+0000 7fba64ff9640 1 -- 172.21.15.97:0/1161598903 <== osd.1 v2:172.21.15.97:6808/4164988844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fba580070c0 con 0x7fba54072420 2024-03-29T04:09:40.331 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:09:40.331 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:09:40.332 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:09:40.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.352+0000 7fba6e4c1640 1 -- 172.21.15.97:0/1161598903 >> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] conn(0x7fba54072420 msgr2=0x7fba54074840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.352+0000 7fba6e4c1640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.97:6808/4164988844,v1:172.21.15.97:6809/4164988844] conn(0x7fba54072420 0x7fba54074840 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.352+0000 7fba6e4c1640 1 -- 172.21.15.97:0/1161598903 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fba5406c560 msgr2=0x7fba5406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.352+0000 7fba6e4c1640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fba5406c560 0x7fba5406ea20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0097a0 tx=0x7fba5c03d040 comp rx=0 tx=0).stop 2024-03-29T04:09:40.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.352+0000 7fba6e4c1640 1 -- 172.21.15.97:0/1161598903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba6815e7b0 msgr2=0x7fba681451f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.352+0000 7fba6e4c1640 1 --2- 172.21.15.97:0/1161598903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba6815e7b0 0x7fba681451f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fba6000efc0 tx=0x7fba6000c490 comp rx=0 tx=0).stop 2024-03-29T04:09:40.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.352+0000 7fba6ca5e640 1 -- 172.21.15.97:0/1161598903 reap_dead start 2024-03-29T04:09:40.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.352+0000 7fba6e4c1640 1 -- 172.21.15.97:0/1161598903 shutdown_connections 2024-03-29T04:09:40.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.352+0000 7fba6e4c1640 1 -- 172.21.15.97:0/1161598903 >> 172.21.15.97:0/1161598903 conn(0x7fba6806b6f0 msgr2=0x7fba680cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.352+0000 7fba6e4c1640 1 -- 172.21.15.97:0/1161598903 shutdown_connections 2024-03-29T04:09:40.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.352+0000 7fba6e4c1640 1 -- 172.21.15.97:0/1161598903 wait complete. 2024-03-29T04:09:40.342 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-29T04:09:40.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.467+0000 7f0908eb8640 1 Processor -- start 2024-03-29T04:09:40.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.467+0000 7f0908eb8640 1 -- start start 2024-03-29T04:09:40.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.467+0000 7f0908eb8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040cdea0 0x7f09040cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.467+0000 7f0908eb8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09040d1310 con 0x7f09040d0b30 2024-03-29T04:09:40.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.467+0000 7f0908eb8640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f090409ac40 con 0x7f090415c740 2024-03-29T04:09:40.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.467+0000 7f0908eb8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f090409b850 con 0x7f09040cdea0 2024-03-29T04:09:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.468+0000 7f08fbfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f09040d0b30 0x7f090406fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:57414/0 (socket says 172.21.15.97:57414) 2024-03-29T04:09:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.468+0000 7f08fbfff640 1 -- 172.21.15.97:0/3496644084 learned_addr learned my addr 172.21.15.97:0/3496644084 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.468+0000 7f0902d9d640 1 --2- 172.21.15.97:0/3496644084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040cdea0 0x7f09040cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.468+0000 7f0901d9b640 1 -- 172.21.15.97:0/3496644084 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1355330653 0 0) 0x7f09040d1310 con 0x7f09040d0b30 2024-03-29T04:09:40.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.468+0000 7f0901d9b640 1 -- 172.21.15.97:0/3496644084 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08f0003520 con 0x7f09040d0b30 2024-03-29T04:09:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.468+0000 7f0901d9b640 1 -- 172.21.15.97:0/3496644084 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1300078814 0 0) 0x7f090409ac40 con 0x7f090415c740 2024-03-29T04:09:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.468+0000 7f0901d9b640 1 -- 172.21.15.97:0/3496644084 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09040d1310 con 0x7f090415c740 2024-03-29T04:09:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.468+0000 7f0902d9d640 1 -- 172.21.15.97:0/3496644084 >> v1:172.21.15.154:6789/0 conn(0x7f090415c740 legacy=0x7f09040d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.468+0000 7f0902d9d640 1 -- 172.21.15.97:0/3496644084 >> v1:172.21.15.97:6789/0 conn(0x7f09040d0b30 legacy=0x7f090406fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.468+0000 7f0902d9d640 1 -- 172.21.15.97:0/3496644084 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090409af20 con 0x7f09040cdea0 2024-03-29T04:09:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.469+0000 7f0902d9d640 1 --2- 172.21.15.97:0/3496644084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040cdea0 0x7f09040cceb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009dc0 tx=0x7f08fc02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea6f2634a4e7023 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.469+0000 7f0901d9b640 1 -- 172.21.15.97:0/3496644084 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc039070 con 0x7f09040cdea0 2024-03-29T04:09:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.469+0000 7f0901d9b640 1 -- 172.21.15.97:0/3496644084 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08fc02ccd0 con 0x7f09040cdea0 2024-03-29T04:09:40.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.469+0000 7f0901d9b640 1 -- 172.21.15.97:0/3496644084 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc034640 con 0x7f09040cdea0 2024-03-29T04:09:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.469+0000 7f0908eb8640 1 -- 172.21.15.97:0/3496644084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040cdea0 msgr2=0x7f09040cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.469+0000 7f0908eb8640 1 --2- 172.21.15.97:0/3496644084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040cdea0 0x7f09040cceb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009dc0 tx=0x7f08fc02bf20 comp rx=0 tx=0).stop 2024-03-29T04:09:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.469+0000 7f0908eb8640 1 -- 172.21.15.97:0/3496644084 shutdown_connections 2024-03-29T04:09:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.469+0000 7f0908eb8640 1 --2- 172.21.15.97:0/3496644084 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040cdea0 0x7f09040cceb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.469+0000 7f0908eb8640 1 -- 172.21.15.97:0/3496644084 >> 172.21.15.97:0/3496644084 conn(0x7f09040b3e90 msgr2=0x7f09040b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.469+0000 7f0908eb8640 1 -- 172.21.15.97:0/3496644084 shutdown_connections 2024-03-29T04:09:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.469+0000 7f0908eb8640 1 -- 172.21.15.97:0/3496644084 wait complete. 2024-03-29T04:09:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.471+0000 7f0908eb8640 1 Processor -- start 2024-03-29T04:09:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.471+0000 7f0908eb8640 1 -- start start 2024-03-29T04:09:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.471+0000 7f0908eb8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040cdea0 0x7f090414f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.471+0000 7f0908eb8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040d0b30 0x7f09040b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.471+0000 7f0908eb8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f090415c740 0x7f090415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.471+0000 7f0908eb8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f090409bce0 con 0x7f09040cdea0 2024-03-29T04:09:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.471+0000 7f0908eb8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f090409b4e0 con 0x7f090415c740 2024-03-29T04:09:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.471+0000 7f0908eb8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f0904070120 con 0x7f09040d0b30 2024-03-29T04:09:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f090259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040cdea0 0x7f090414f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f090259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040cdea0 0x7f090414f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36338/0 (socket says 172.21.15.97:36338) 2024-03-29T04:09:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f090259c640 1 -- 172.21.15.97:0/3261195741 learned_addr learned my addr 172.21.15.97:0/3261195741 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f0902d9d640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f090415c740 0x7f090415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f08fbfff640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040d0b30 0x7f09040b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f090259c640 1 -- 172.21.15.97:0/3261195741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040d0b30 msgr2=0x7f09040b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f090259c640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040d0b30 0x7f09040b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f090259c640 1 -- 172.21.15.97:0/3261195741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f090415c740 msgr2=0x7f090415a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f090259c640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f090415c740 0x7f090415a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f090259c640 1 -- 172.21.15.97:0/3261195741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09040b3950 con 0x7f09040cdea0 2024-03-29T04:09:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f090259c640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040cdea0 0x7f090414f860 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f08f400c430 tx=0x7f08f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f08faffd640 1 -- 172.21.15.97:0/3261195741 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4017070 con 0x7f09040cdea0 2024-03-29T04:09:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f08faffd640 1 -- 172.21.15.97:0/3261195741 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08f40052f0 con 0x7f09040cdea0 2024-03-29T04:09:40.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f08faffd640 1 -- 172.21.15.97:0/3261195741 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4003830 con 0x7f09040cdea0 2024-03-29T04:09:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f0908eb8640 1 -- 172.21.15.97:0/3261195741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09040b3390 con 0x7f09040cdea0 2024-03-29T04:09:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.472+0000 7f0908eb8640 1 -- 172.21.15.97:0/3261195741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f090415d660 con 0x7f09040cdea0 2024-03-29T04:09:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.473+0000 7f0908eb8640 1 -- 172.21.15.97:0/3261195741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09040d1aa0 con 0x7f09040cdea0 2024-03-29T04:09:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.474+0000 7f08faffd640 1 -- 172.21.15.97:0/3261195741 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f08f401a070 con 0x7f09040cdea0 2024-03-29T04:09:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.474+0000 7f08faffd640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f08e406c530 0x7f08e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.474+0000 7f08faffd640 1 -- 172.21.15.97:0/3261195741 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f08f408a5d0 con 0x7f09040cdea0 2024-03-29T04:09:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.474+0000 7f08fbfff640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f08e406c530 0x7f08e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.474+0000 7f08fbfff640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f08e406c530 0x7f08e406e9f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f09040b3180 tx=0x7f08ec007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.482+0000 7f08faffd640 1 -- 172.21.15.97:0/3261195741 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f09040d1aa0 con 0x7f09040cdea0 2024-03-29T04:09:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.660+0000 7f0908eb8640 1 -- 172.21.15.97:0/3261195741 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f090414fda0 con 0x7f08e406c530 2024-03-29T04:09:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.660+0000 7f08faffd640 1 -- 172.21.15.97:0/3261195741 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f090414fda0 con 0x7f08e406c530 2024-03-29T04:09:40.640 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 1 to deep-scrub 2024-03-29T04:09:40.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.664+0000 7f0908eb8640 1 -- 172.21.15.97:0/3261195741 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f08e406c530 msgr2=0x7f08e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.664+0000 7f0908eb8640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f08e406c530 0x7f08e406e9f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f09040b3180 tx=0x7f08ec007d30 comp rx=0 tx=0).stop 2024-03-29T04:09:40.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.664+0000 7f0908eb8640 1 -- 172.21.15.97:0/3261195741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040cdea0 msgr2=0x7f090414f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.664+0000 7f0908eb8640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040cdea0 0x7f090414f860 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f08f400c430 tx=0x7f08f400c8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.664+0000 7f0908eb8640 1 -- 172.21.15.97:0/3261195741 shutdown_connections 2024-03-29T04:09:40.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.664+0000 7f0908eb8640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f08e406c530 0x7f08e406e9f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.664+0000 7f0908eb8640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f090415c740 0x7f090415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.664+0000 7f0908eb8640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09040d0b30 0x7f09040b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.664+0000 7f0908eb8640 1 --2- 172.21.15.97:0/3261195741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040cdea0 0x7f090414f860 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.664+0000 7f0908eb8640 1 -- 172.21.15.97:0/3261195741 >> 172.21.15.97:0/3261195741 conn(0x7f09040b3e90 msgr2=0x7f09040b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.664+0000 7f0908eb8640 1 -- 172.21.15.97:0/3261195741 shutdown_connections 2024-03-29T04:09:40.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.664+0000 7f0908eb8640 1 -- 172.21.15.97:0/3261195741 wait complete. 2024-03-29T04:09:40.658 INFO:tasks.ceph:Scrubbing osd.2 2024-03-29T04:09:40.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:09:40.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.784+0000 7f185f84a640 1 Processor -- start 2024-03-29T04:09:40.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.784+0000 7f185f84a640 1 -- start start 2024-03-29T04:09:40.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.785+0000 7f185f84a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f185815dc80 0x7f18580cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.785+0000 7f185f84a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f185809bc20 0x7f18580cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.785+0000 7f185f84a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18580714e0 con 0x7f185806fa20 2024-03-29T04:09:40.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.785+0000 7f185f84a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f18580cebf0 con 0x7f185815dc80 2024-03-29T04:09:40.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.785+0000 7f185f84a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f18580717c0 con 0x7f185809bc20 2024-03-29T04:09:40.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.785+0000 7f185d5e6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f185815dc80 0x7f18580cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.785+0000 7f185d5e6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f185815dc80 0x7f18580cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39390/0 (socket says 172.21.15.97:39390) 2024-03-29T04:09:40.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.785+0000 7f185dde7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f185809bc20 0x7f18580cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.785+0000 7f185d5e6640 1 -- 172.21.15.97:0/3769656241 learned_addr learned my addr 172.21.15.97:0/3769656241 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.785+0000 7f185d5e6640 1 -- 172.21.15.97:0/3769656241 >> v2:172.21.15.154:3300/0 conn(0x7f185809bc20 msgr2=0x7f18580cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.785+0000 7f185d5e6640 1 --2- 172.21.15.97:0/3769656241 >> v2:172.21.15.154:3300/0 conn(0x7f185809bc20 0x7f18580cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.785+0000 7f185d5e6640 1 -- 172.21.15.97:0/3769656241 >> v1:172.21.15.154:6789/0 conn(0x7f185806fa20 legacy=0x7f185809b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.785+0000 7f185d5e6640 1 -- 172.21.15.97:0/3769656241 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185809aa30 con 0x7f185815dc80 2024-03-29T04:09:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.786+0000 7f185d5e6640 1 --2- 172.21.15.97:0/3769656241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185815dc80 0x7f18580cffe0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1858073720 tx=0x7f184802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9649ad0b28355a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.786+0000 7f184ffff640 1 -- 172.21.15.97:0/3769656241 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848035020 con 0x7f185815dc80 2024-03-29T04:09:40.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.786+0000 7f184ffff640 1 -- 172.21.15.97:0/3769656241 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1848031870 con 0x7f185815dc80 2024-03-29T04:09:40.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.786+0000 7f184ffff640 1 -- 172.21.15.97:0/3769656241 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848031c40 con 0x7f185815dc80 2024-03-29T04:09:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.787+0000 7f185f84a640 1 -- 172.21.15.97:0/3769656241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185815dc80 msgr2=0x7f18580cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.787+0000 7f185f84a640 1 --2- 172.21.15.97:0/3769656241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185815dc80 0x7f18580cffe0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1858073720 tx=0x7f184802b910 comp rx=0 tx=0).stop 2024-03-29T04:09:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.787+0000 7f185f84a640 1 -- 172.21.15.97:0/3769656241 shutdown_connections 2024-03-29T04:09:40.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.787+0000 7f185f84a640 1 --2- 172.21.15.97:0/3769656241 >> v2:172.21.15.154:3300/0 conn(0x7f185809bc20 0x7f18580cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.787+0000 7f185f84a640 1 --2- 172.21.15.97:0/3769656241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185815dc80 0x7f18580cffe0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.787+0000 7f185f84a640 1 -- 172.21.15.97:0/3769656241 >> 172.21.15.97:0/3769656241 conn(0x7f18580b3f00 msgr2=0x7f18580b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.787+0000 7f185f84a640 1 -- 172.21.15.97:0/3769656241 shutdown_connections 2024-03-29T04:09:40.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.787+0000 7f185f84a640 1 -- 172.21.15.97:0/3769656241 wait complete. 2024-03-29T04:09:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.788+0000 7f185f84a640 1 Processor -- start 2024-03-29T04:09:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.788+0000 7f185f84a640 1 -- start start 2024-03-29T04:09:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185f84a640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f185806fa20 0x7f18580c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185f84a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185809bc20 0x7f1858154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185f84a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f185815dc80 0x7f18580cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185f84a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f185806feb0 con 0x7f185809bc20 2024-03-29T04:09:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185f84a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f185809aff0 con 0x7f185815dc80 2024-03-29T04:09:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185f84a640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1858070030 con 0x7f185806fa20 2024-03-29T04:09:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185cde5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185809bc20 0x7f1858154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185cde5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185809bc20 0x7f1858154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39394/0 (socket says 172.21.15.97:39394) 2024-03-29T04:09:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185cde5640 1 -- 172.21.15.97:0/4246119716 learned_addr learned my addr 172.21.15.97:0/4246119716 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185dde7640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f185815dc80 0x7f18580cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185cde5640 1 -- 172.21.15.97:0/4246119716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f185806fa20 msgr2=0x7f18580c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185d5e6640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f185806fa20 0x7f18580c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185cde5640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f185806fa20 0x7f18580c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185cde5640 1 -- 172.21.15.97:0/4246119716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f185815dc80 msgr2=0x7f18580cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185cde5640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f185815dc80 0x7f18580cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185cde5640 1 -- 172.21.15.97:0/4246119716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18580b6090 con 0x7f185809bc20 2024-03-29T04:09:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.789+0000 7f185dde7640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f185815dc80 0x7f18580cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.790+0000 7f185cde5640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185809bc20 0x7f1858154960 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f185400c480 tx=0x7f185400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.790+0000 7f184dffb640 1 -- 172.21.15.97:0/4246119716 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18540052d0 con 0x7f185809bc20 2024-03-29T04:09:40.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.790+0000 7f184dffb640 1 -- 172.21.15.97:0/4246119716 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1854005470 con 0x7f185809bc20 2024-03-29T04:09:40.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.790+0000 7f185f84a640 1 -- 172.21.15.97:0/4246119716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18580c9be0 con 0x7f185809bc20 2024-03-29T04:09:40.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.790+0000 7f184dffb640 1 -- 172.21.15.97:0/4246119716 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854012870 con 0x7f185809bc20 2024-03-29T04:09:40.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.790+0000 7f185f84a640 1 -- 172.21.15.97:0/4246119716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18580ca480 con 0x7f185809bc20 2024-03-29T04:09:40.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.791+0000 7f185f84a640 1 -- 172.21.15.97:0/4246119716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1858068640 con 0x7f185809bc20 2024-03-29T04:09:40.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.792+0000 7f184dffb640 1 -- 172.21.15.97:0/4246119716 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1854012a10 con 0x7f185809bc20 2024-03-29T04:09:40.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.792+0000 7f184dffb640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f183c06c530 0x7f183c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.792+0000 7f184dffb640 1 -- 172.21.15.97:0/4246119716 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f185408ba80 con 0x7f185809bc20 2024-03-29T04:09:40.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.792+0000 7f185d5e6640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f183c06c530 0x7f183c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.792+0000 7f184dffb640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] conn(0x7f183c0723f0 0x7f183c074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.792+0000 7f184dffb640 1 -- 172.21.15.97:0/4246119716 --> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1854010e50 con 0x7f183c0723f0 2024-03-29T04:09:40.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.792+0000 7f184dffb640 1 -- 172.21.15.97:0/4246119716 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f185408bf10 con 0x7f185809bc20 2024-03-29T04:09:40.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.792+0000 7f185dde7640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] conn(0x7f183c0723f0 0x7f183c074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.793+0000 7f185d5e6640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f183c06c530 0x7f183c06e9f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1848036000 tx=0x7f1848008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.793+0000 7f185dde7640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] conn(0x7f183c0723f0 0x7f183c074810 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.794+0000 7f184dffb640 1 -- 172.21.15.97:0/4246119716 <== osd.2 v2:172.21.15.97:6816/127556154 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1854010e50 con 0x7f183c0723f0 2024-03-29T04:09:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.806+0000 7f185f84a640 1 -- 172.21.15.97:0/4246119716 --> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f18580b4ab0 con 0x7f183c0723f0 2024-03-29T04:09:40.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.807+0000 7f184dffb640 1 -- 172.21.15.97:0/4246119716 <== osd.2 v2:172.21.15.97:6816/127556154 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f18580b4ab0 con 0x7f183c0723f0 2024-03-29T04:09:40.786 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:09:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:09:40.787 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:09:40.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.807+0000 7f185f84a640 1 -- 172.21.15.97:0/4246119716 >> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] conn(0x7f183c0723f0 msgr2=0x7f183c074810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.807+0000 7f185f84a640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.97:6816/127556154,v1:172.21.15.97:6817/127556154] conn(0x7f183c0723f0 0x7f183c074810 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.807+0000 7f185f84a640 1 -- 172.21.15.97:0/4246119716 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f183c06c530 msgr2=0x7f183c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.807+0000 7f185f84a640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f183c06c530 0x7f183c06e9f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1848036000 tx=0x7f1848008030 comp rx=0 tx=0).stop 2024-03-29T04:09:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.807+0000 7f185f84a640 1 -- 172.21.15.97:0/4246119716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185809bc20 msgr2=0x7f1858154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.807+0000 7f185f84a640 1 --2- 172.21.15.97:0/4246119716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f185809bc20 0x7f1858154960 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f185400c480 tx=0x7f185400c940 comp rx=0 tx=0).stop 2024-03-29T04:09:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.807+0000 7f185dde7640 1 -- 172.21.15.97:0/4246119716 reap_dead start 2024-03-29T04:09:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.807+0000 7f185f84a640 1 -- 172.21.15.97:0/4246119716 shutdown_connections 2024-03-29T04:09:40.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.807+0000 7f185f84a640 1 -- 172.21.15.97:0/4246119716 >> 172.21.15.97:0/4246119716 conn(0x7f18580b3f00 msgr2=0x7f18581584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.807+0000 7f185f84a640 1 -- 172.21.15.97:0/4246119716 shutdown_connections 2024-03-29T04:09:40.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.807+0000 7f185f84a640 1 -- 172.21.15.97:0/4246119716 wait complete. 2024-03-29T04:09:40.797 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-29T04:09:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.920+0000 7fd2d2b96640 1 Processor -- start 2024-03-29T04:09:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.920+0000 7fd2d2b96640 1 -- start start 2024-03-29T04:09:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.920+0000 7fd2d2b96640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd2cc0d0b30 0x7fd2cc0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.920+0000 7fd2d2b96640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2cc0cceb0 0x7fd2cc06fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.920+0000 7fd2d2b96640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2cc0d1310 con 0x7fd2cc15c740 2024-03-29T04:09:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.921+0000 7fd2d2b96640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fd2cc09b850 con 0x7fd2cc0d0b30 2024-03-29T04:09:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.921+0000 7fd2d2b96640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd2cc09ac40 con 0x7fd2cc0cceb0 2024-03-29T04:09:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.921+0000 7fd2d1133640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2cc0cceb0 0x7fd2cc06fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.921+0000 7fd2cbfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd2cc0d0b30 0x7fd2cc0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.921+0000 7fd2d0932640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd2cc15c740 0x7fd2cc0d0420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.97:56106/0 (socket says 172.21.15.97:56106) 2024-03-29T04:09:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.921+0000 7fd2d0932640 1 -- 172.21.15.97:0/505598899 learned_addr learned my addr 172.21.15.97:0/505598899 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.921+0000 7fd2cb7fe640 1 -- 172.21.15.97:0/505598899 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1645311032 0 0) 0x7fd2cc0d1310 con 0x7fd2cc15c740 2024-03-29T04:09:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.921+0000 7fd2d1133640 1 -- 172.21.15.97:0/505598899 >> v2:172.21.15.154:3300/0 conn(0x7fd2cc0d0b30 msgr2=0x7fd2cc0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.921+0000 7fd2d1133640 1 --2- 172.21.15.97:0/505598899 >> v2:172.21.15.154:3300/0 conn(0x7fd2cc0d0b30 0x7fd2cc0cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.921+0000 7fd2d1133640 1 -- 172.21.15.97:0/505598899 >> v1:172.21.15.154:6789/0 conn(0x7fd2cc15c740 legacy=0x7fd2cc0d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.921+0000 7fd2d1133640 1 -- 172.21.15.97:0/505598899 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc09af20 con 0x7fd2cc0cceb0 2024-03-29T04:09:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.921+0000 7fd2cbfff640 1 --2- 172.21.15.97:0/505598899 >> v2:172.21.15.154:3300/0 conn(0x7fd2cc0d0b30 0x7fd2cc0cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.922+0000 7fd2d1133640 1 --2- 172.21.15.97:0/505598899 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2cc0cceb0 0x7fd2cc06fa10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009d90 tx=0x7fd2c402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5cce066641018cd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.922+0000 7fd2cb7fe640 1 -- 172.21.15.97:0/505598899 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4039070 con 0x7fd2cc0cceb0 2024-03-29T04:09:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.923+0000 7fd2cb7fe640 1 -- 172.21.15.97:0/505598899 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c402cc50 con 0x7fd2cc0cceb0 2024-03-29T04:09:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.923+0000 7fd2cb7fe640 1 -- 172.21.15.97:0/505598899 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4034440 con 0x7fd2cc0cceb0 2024-03-29T04:09:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.923+0000 7fd2d2b96640 1 -- 172.21.15.97:0/505598899 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2cc0cceb0 msgr2=0x7fd2cc06fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.923+0000 7fd2d2b96640 1 --2- 172.21.15.97:0/505598899 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2cc0cceb0 0x7fd2cc06fa10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009d90 tx=0x7fd2c402bea0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.923+0000 7fd2d2b96640 1 -- 172.21.15.97:0/505598899 shutdown_connections 2024-03-29T04:09:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.923+0000 7fd2d2b96640 1 --2- 172.21.15.97:0/505598899 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2cc0cceb0 0x7fd2cc06fa10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.923+0000 7fd2d2b96640 1 --2- 172.21.15.97:0/505598899 >> v2:172.21.15.154:3300/0 conn(0x7fd2cc0d0b30 0x7fd2cc0cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.923+0000 7fd2d2b96640 1 -- 172.21.15.97:0/505598899 >> 172.21.15.97:0/505598899 conn(0x7fd2cc0b3e90 msgr2=0x7fd2cc0b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.923+0000 7fd2d2b96640 1 -- 172.21.15.97:0/505598899 shutdown_connections 2024-03-29T04:09:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.923+0000 7fd2d2b96640 1 -- 172.21.15.97:0/505598899 wait complete. 2024-03-29T04:09:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.924+0000 7fd2d2b96640 1 Processor -- start 2024-03-29T04:09:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.924+0000 7fd2d2b96640 1 -- start start 2024-03-29T04:09:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.925+0000 7fd2d2b96640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2cc0cceb0 0x7fd2cc0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.925+0000 7fd2d2b96640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2cc0d0b30 0x7fd2cc15b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.925+0000 7fd2d2b96640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2cc15c740 0x7fd2cc0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.925+0000 7fd2d2b96640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc09bce0 con 0x7fd2cc0d0b30 2024-03-29T04:09:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.925+0000 7fd2d2b96640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc09b4e0 con 0x7fd2cc0cceb0 2024-03-29T04:09:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.925+0000 7fd2d2b96640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fd2cc06ff50 con 0x7fd2cc15c740 2024-03-29T04:09:40.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.925+0000 7fd2cbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2cc0d0b30 0x7fd2cc15b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.925+0000 7fd2cbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2cc0d0b30 0x7fd2cc15b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39398/0 (socket says 172.21.15.97:39398) 2024-03-29T04:09:40.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.925+0000 7fd2cbfff640 1 -- 172.21.15.97:0/959317745 learned_addr learned my addr 172.21.15.97:0/959317745 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.925+0000 7fd2d0932640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2cc0cceb0 0x7fd2cc0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2d1133640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2cc15c740 0x7fd2cc0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2cbfff640 1 -- 172.21.15.97:0/959317745 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2cc15c740 msgr2=0x7fd2cc0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2cbfff640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2cc15c740 0x7fd2cc0cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2cbfff640 1 -- 172.21.15.97:0/959317745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2cc0cceb0 msgr2=0x7fd2cc0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2cbfff640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2cc0cceb0 0x7fd2cc0c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2cbfff640 1 -- 172.21.15.97:0/959317745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc0b7fd0 con 0x7fd2cc0d0b30 2024-03-29T04:09:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2d1133640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2cc15c740 0x7fd2cc0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2d0932640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2cc0cceb0 0x7fd2cc0c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2cbfff640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2cc0d0b30 0x7fd2cc15b940 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800ec10 tx=0x7fd2b800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2c97fa640 1 -- 172.21.15.97:0/959317745 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8017070 con 0x7fd2cc0d0b30 2024-03-29T04:09:40.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2c97fa640 1 -- 172.21.15.97:0/959317745 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b800cc30 con 0x7fd2cc0d0b30 2024-03-29T04:09:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2d2b96640 1 -- 172.21.15.97:0/959317745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2cc0c7050 con 0x7fd2cc0d0b30 2024-03-29T04:09:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2c97fa640 1 -- 172.21.15.97:0/959317745 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8013470 con 0x7fd2cc0d0b30 2024-03-29T04:09:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.926+0000 7fd2d2b96640 1 -- 172.21.15.97:0/959317745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2cc0c7610 con 0x7fd2cc0d0b30 2024-03-29T04:09:40.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.927+0000 7fd2d2b96640 1 -- 172.21.15.97:0/959317745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2cc068d50 con 0x7fd2cc0d0b30 2024-03-29T04:09:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.927+0000 7fd2c97fa640 1 -- 172.21.15.97:0/959317745 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd2b801a070 con 0x7fd2cc0d0b30 2024-03-29T04:09:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.928+0000 7fd2c97fa640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd2b006c530 0x7fd2b006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.928+0000 7fd2c97fa640 1 -- 172.21.15.97:0/959317745 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fd2b808b6b0 con 0x7fd2cc0d0b30 2024-03-29T04:09:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.928+0000 7fd2d0932640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd2b006c530 0x7fd2b006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.928+0000 7fd2d0932640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd2b006c530 0x7fd2b006e9f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0000f60 tx=0x7fd2c0007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:40.932+0000 7fd2c97fa640 1 -- 172.21.15.97:0/959317745 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd2b8059fe0 con 0x7fd2cc0d0b30 2024-03-29T04:09:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.123+0000 7fd2d2b96640 1 -- 172.21.15.97:0/959317745 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fd2cc0b82b0 con 0x7fd2b006c530 2024-03-29T04:09:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.123+0000 7fd2c97fa640 1 -- 172.21.15.97:0/959317745 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd2cc0b82b0 con 0x7fd2b006c530 2024-03-29T04:09:41.103 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 2 to deep-scrub 2024-03-29T04:09:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.125+0000 7fd2d2b96640 1 -- 172.21.15.97:0/959317745 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd2b006c530 msgr2=0x7fd2b006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.125+0000 7fd2d2b96640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd2b006c530 0x7fd2b006e9f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0000f60 tx=0x7fd2c0007ca0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.125+0000 7fd2d2b96640 1 -- 172.21.15.97:0/959317745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2cc0d0b30 msgr2=0x7fd2cc15b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.125+0000 7fd2d2b96640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2cc0d0b30 0x7fd2cc15b940 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800ec10 tx=0x7fd2b800c520 comp rx=0 tx=0).stop 2024-03-29T04:09:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.125+0000 7fd2d2b96640 1 -- 172.21.15.97:0/959317745 shutdown_connections 2024-03-29T04:09:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.125+0000 7fd2d2b96640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fd2b006c530 0x7fd2b006e9f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.125+0000 7fd2d2b96640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2cc15c740 0x7fd2cc0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.125+0000 7fd2d2b96640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2cc0d0b30 0x7fd2cc15b940 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.125+0000 7fd2d2b96640 1 --2- 172.21.15.97:0/959317745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2cc0cceb0 0x7fd2cc0c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.125+0000 7fd2d2b96640 1 -- 172.21.15.97:0/959317745 >> 172.21.15.97:0/959317745 conn(0x7fd2cc0b3e90 msgr2=0x7fd2cc070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.125+0000 7fd2d2b96640 1 -- 172.21.15.97:0/959317745 shutdown_connections 2024-03-29T04:09:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.125+0000 7fd2d2b96640 1 -- 172.21.15.97:0/959317745 wait complete. 2024-03-29T04:09:41.116 INFO:tasks.ceph:Scrubbing osd.3 2024-03-29T04:09:41.116 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:09:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.241+0000 7f4a48a0d640 1 Processor -- start 2024-03-29T04:09:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.241+0000 7f4a48a0d640 1 -- start start 2024-03-29T04:09:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.242+0000 7f4a48a0d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4a4406b510 0x7f4a4406b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.242+0000 7f4a48a0d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440b2c10 0x7f4a440b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.242+0000 7f4a48a0d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a440ce810 con 0x7f4a440d1100 2024-03-29T04:09:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.242+0000 7f4a48a0d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f4a4406be30 con 0x7f4a4406b510 2024-03-29T04:09:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.242+0000 7f4a48a0d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4a440b6420 con 0x7f4a440b2c10 2024-03-29T04:09:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.242+0000 7f4a42d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4a440d1100 0x7f4a440d1590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41612/0 (socket says 172.21.15.97:41612) 2024-03-29T04:09:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.242+0000 7f4a42d9d640 1 -- 172.21.15.97:0/1926895335 learned_addr learned my addr 172.21.15.97:0/1926895335 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:41.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3558329521 0 0) 0x7f4a440ce810 con 0x7f4a440d1100 2024-03-29T04:09:41.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4259c640 1 --2- 172.21.15.97:0/1926895335 >> v2:172.21.15.154:3300/0 conn(0x7f4a4406b510 0x7f4a4406b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a41d9b640 1 --2- 172.21.15.97:0/1926895335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440b2c10 0x7f4a440b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a2c003520 con 0x7f4a440d1100 2024-03-29T04:09:41.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1593764252 0 0) 0x7f4a2c003520 con 0x7f4a440d1100 2024-03-29T04:09:41.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440b2c10 msgr2=0x7f4a440b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:41.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 --2- 172.21.15.97:0/1926895335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440b2c10 0x7f4a440b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 >> v2:172.21.15.154:3300/0 conn(0x7f4a4406b510 msgr2=0x7f4a4406b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:41.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 --2- 172.21.15.97:0/1926895335 >> v2:172.21.15.154:3300/0 conn(0x7f4a4406b510 0x7f4a4406b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a440b6700 con 0x7f4a440d1100 2024-03-29T04:09:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f4a3c001670 con 0x7f4a440d1100 2024-03-29T04:09:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 >> v1:172.21.15.97:6789/0 conn(0x7f4a440d1100 legacy=0x7f4a440d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 --2- 172.21.15.97:0/1926895335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0039b0 0x7f4a2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 --2- 172.21.15.97:0/1926895335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a2c004800 0x7f4a2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 --2- 172.21.15.97:0/1926895335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440b2c10 0x7f4a2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4406be30 con 0x7f4a2c0039b0 2024-03-29T04:09:41.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4a440b6420 con 0x7f4a2c004800 2024-03-29T04:09:41.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2c00b9c0 con 0x7f4a440b2c10 2024-03-29T04:09:41.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4259c640 1 --2- 172.21.15.97:0/1926895335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0039b0 0x7f4a2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a42d9d640 1 --2- 172.21.15.97:0/1926895335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440b2c10 0x7f4a2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a41d9b640 1 --2- 172.21.15.97:0/1926895335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a2c004800 0x7f4a2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4259c640 1 -- 172.21.15.97:0/1926895335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440b2c10 msgr2=0x7f4a2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:41.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4259c640 1 --2- 172.21.15.97:0/1926895335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440b2c10 0x7f4a2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4259c640 1 -- 172.21.15.97:0/1926895335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a2c004800 msgr2=0x7f4a2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:41.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4259c640 1 --2- 172.21.15.97:0/1926895335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a2c004800 0x7f4a2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a4259c640 1 -- 172.21.15.97:0/1926895335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2c003520 con 0x7f4a2c0039b0 2024-03-29T04:09:41.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.243+0000 7f4a42d9d640 1 --2- 172.21.15.97:0/1926895335 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440b2c10 0x7f4a2c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:41.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.244+0000 7f4a4259c640 1 --2- 172.21.15.97:0/1926895335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0039b0 0x7f4a2c003dd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400e170 tx=0x7f4a2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861df9369b33425e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.244+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34039070 con 0x7f4a2c0039b0 2024-03-29T04:09:41.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.244+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a3402cc20 con 0x7f4a2c0039b0 2024-03-29T04:09:41.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.244+0000 7f4a4159a640 1 -- 172.21.15.97:0/1926895335 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34009c10 con 0x7f4a2c0039b0 2024-03-29T04:09:41.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.245+0000 7f4a48a0d640 1 -- 172.21.15.97:0/1926895335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0039b0 msgr2=0x7f4a2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.245+0000 7f4a48a0d640 1 --2- 172.21.15.97:0/1926895335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0039b0 0x7f4a2c003dd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400e170 tx=0x7f4a2c005830 comp rx=0 tx=0).stop 2024-03-29T04:09:41.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.245+0000 7f4a42d9d640 1 -- 172.21.15.97:0/1926895335 reap_dead start 2024-03-29T04:09:41.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.245+0000 7f4a48a0d640 1 -- 172.21.15.97:0/1926895335 shutdown_connections 2024-03-29T04:09:41.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.245+0000 7f4a48a0d640 1 -- 172.21.15.97:0/1926895335 >> 172.21.15.97:0/1926895335 conn(0x7f4a440b6af0 msgr2=0x7f4a4409aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.245+0000 7f4a48a0d640 1 -- 172.21.15.97:0/1926895335 shutdown_connections 2024-03-29T04:09:41.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.245+0000 7f4a48a0d640 1 -- 172.21.15.97:0/1926895335 wait complete. 2024-03-29T04:09:41.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.246+0000 7f4a48a0d640 1 Processor -- start 2024-03-29T04:09:41.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.246+0000 7f4a48a0d640 1 -- start start 2024-03-29T04:09:41.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a48a0d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0039b0 0x7f4a44140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a48a0d640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440ca730 0x7f4a44143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a48a0d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a440cabb0 0x7f4a44143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a48a0d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44145430 con 0x7f4a2c0039b0 2024-03-29T04:09:41.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a48a0d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4a441452b0 con 0x7f4a440cabb0 2024-03-29T04:09:41.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a48a0d640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f4a440b6040 con 0x7f4a440ca730 2024-03-29T04:09:41.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a4259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0039b0 0x7f4a44140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a4259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0039b0 0x7f4a44140060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39408/0 (socket says 172.21.15.97:39408) 2024-03-29T04:09:41.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a4259c640 1 -- 172.21.15.97:0/224845092 learned_addr learned my addr 172.21.15.97:0/224845092 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:41.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a42d9d640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a440cabb0 0x7f4a44143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a41d9b640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440ca730 0x7f4a44143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a4259c640 1 -- 172.21.15.97:0/224845092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440ca730 msgr2=0x7f4a44143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a4259c640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440ca730 0x7f4a44143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a4259c640 1 -- 172.21.15.97:0/224845092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a440cabb0 msgr2=0x7f4a44143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a4259c640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a440cabb0 0x7f4a44143b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a4259c640 1 -- 172.21.15.97:0/224845092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a441405a0 con 0x7f4a2c0039b0 2024-03-29T04:09:41.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a41d9b640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a440ca730 0x7f4a44143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:41.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a4259c640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0039b0 0x7f4a44140060 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400e170 tx=0x7f4a34009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.247+0000 7f4a32ffd640 1 -- 172.21.15.97:0/224845092 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34039050 con 0x7f4a2c0039b0 2024-03-29T04:09:41.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.248+0000 7f4a32ffd640 1 -- 172.21.15.97:0/224845092 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a34045070 con 0x7f4a2c0039b0 2024-03-29T04:09:41.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.248+0000 7f4a48a0d640 1 -- 172.21.15.97:0/224845092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4415fd60 con 0x7f4a2c0039b0 2024-03-29T04:09:41.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.248+0000 7f4a32ffd640 1 -- 172.21.15.97:0/224845092 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34009c10 con 0x7f4a2c0039b0 2024-03-29T04:09:41.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.248+0000 7f4a48a0d640 1 -- 172.21.15.97:0/224845092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a44160320 con 0x7f4a2c0039b0 2024-03-29T04:09:41.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.248+0000 7f4a48a0d640 1 -- 172.21.15.97:0/224845092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a441609e0 con 0x7f4a2c0039b0 2024-03-29T04:09:41.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.249+0000 7f4a32ffd640 1 -- 172.21.15.97:0/224845092 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4a34005d10 con 0x7f4a2c0039b0 2024-03-29T04:09:41.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.249+0000 7f4a32ffd640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4a2406c530 0x7f4a2406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.249+0000 7f4a32ffd640 1 -- 172.21.15.97:0/224845092 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f4a340adac0 con 0x7f4a2c0039b0 2024-03-29T04:09:41.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.249+0000 7f4a41d9b640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4a2406c530 0x7f4a2406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.249+0000 7f4a32ffd640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] conn(0x7f4a240723f0 0x7f4a24074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.249+0000 7f4a32ffd640 1 -- 172.21.15.97:0/224845092 --> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a3402cc20 con 0x7f4a240723f0 2024-03-29T04:09:41.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.249+0000 7f4a32ffd640 1 -- 172.21.15.97:0/224845092 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4a340b22c0 con 0x7f4a2c0039b0 2024-03-29T04:09:41.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.250+0000 7f4a42d9d640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] conn(0x7f4a240723f0 0x7f4a24074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.250+0000 7f4a41d9b640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4a2406c530 0x7f4a2406e9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4a440b2330 tx=0x7f4a38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.250+0000 7f4a42d9d640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] conn(0x7f4a240723f0 0x7f4a24074810 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.250+0000 7f4a32ffd640 1 -- 172.21.15.97:0/224845092 <== osd.3 v2:172.21.15.97:6824/1504398439 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a3402cc20 con 0x7f4a240723f0 2024-03-29T04:09:41.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.263+0000 7f4a48a0d640 1 -- 172.21.15.97:0/224845092 --> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4a44068640 con 0x7f4a240723f0 2024-03-29T04:09:41.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.263+0000 7f4a32ffd640 1 -- 172.21.15.97:0/224845092 <== osd.3 v2:172.21.15.97:6824/1504398439 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4a44068640 con 0x7f4a240723f0 2024-03-29T04:09:41.248 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:09:41.248 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:09:41.248 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:09:41.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.264+0000 7f4a48a0d640 1 -- 172.21.15.97:0/224845092 >> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] conn(0x7f4a240723f0 msgr2=0x7f4a24074810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.264+0000 7f4a48a0d640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.97:6824/1504398439,v1:172.21.15.97:6825/1504398439] conn(0x7f4a240723f0 0x7f4a24074810 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.264+0000 7f4a48a0d640 1 -- 172.21.15.97:0/224845092 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4a2406c530 msgr2=0x7f4a2406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.264+0000 7f4a48a0d640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f4a2406c530 0x7f4a2406e9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4a440b2330 tx=0x7f4a38008040 comp rx=0 tx=0).stop 2024-03-29T04:09:41.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.264+0000 7f4a48a0d640 1 -- 172.21.15.97:0/224845092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0039b0 msgr2=0x7f4a44140060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.264+0000 7f4a48a0d640 1 --2- 172.21.15.97:0/224845092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a2c0039b0 0x7f4a44140060 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400e170 tx=0x7f4a34009750 comp rx=0 tx=0).stop 2024-03-29T04:09:41.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.264+0000 7f4a42d9d640 1 -- 172.21.15.97:0/224845092 reap_dead start 2024-03-29T04:09:41.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.264+0000 7f4a48a0d640 1 -- 172.21.15.97:0/224845092 shutdown_connections 2024-03-29T04:09:41.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.264+0000 7f4a48a0d640 1 -- 172.21.15.97:0/224845092 >> 172.21.15.97:0/224845092 conn(0x7f4a440b6af0 msgr2=0x7f4a440b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.264+0000 7f4a48a0d640 1 -- 172.21.15.97:0/224845092 shutdown_connections 2024-03-29T04:09:41.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.264+0000 7f4a48a0d640 1 -- 172.21.15.97:0/224845092 wait complete. 2024-03-29T04:09:41.254 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-29T04:09:41.257 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:41.298 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:41.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.377+0000 7f1ffaf31640 1 Processor -- start 2024-03-29T04:09:41.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.377+0000 7f1ffaf31640 1 -- start start 2024-03-29T04:09:41.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ffaf31640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1ff4071270 0x7f1ff40cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ffaf31640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ff40cff70 0x7f1ff415e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ffaf31640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff40d14a0 con 0x7f1ff4071650 2024-03-29T04:09:41.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ffaf31640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff40d1010 con 0x7f1ff40cff70 2024-03-29T04:09:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ffaf31640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff40d0400 con 0x7f1ff4071270 2024-03-29T04:09:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ff94ce640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ff40cff70 0x7f1ff415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ff94ce640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ff40cff70 0x7f1ff415e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39420/0 (socket says 172.21.15.97:39420) 2024-03-29T04:09:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ff94ce640 1 -- 172.21.15.97:0/2913086625 learned_addr learned my addr 172.21.15.97:0/2913086625 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ff8ccd640 1 --2- 172.21.15.97:0/2913086625 >> v2:172.21.15.154:3300/0 conn(0x7f1ff4071270 0x7f1ff40cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ff94ce640 1 -- 172.21.15.97:0/2913086625 >> v2:172.21.15.154:3300/0 conn(0x7f1ff4071270 msgr2=0x7f1ff40cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ff94ce640 1 --2- 172.21.15.97:0/2913086625 >> v2:172.21.15.154:3300/0 conn(0x7f1ff4071270 0x7f1ff40cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ff94ce640 1 -- 172.21.15.97:0/2913086625 >> v1:172.21.15.154:6789/0 conn(0x7f1ff4071650 legacy=0x7f1ff4162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ff94ce640 1 -- 172.21.15.97:0/2913086625 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff40d06e0 con 0x7f1ff40cff70 2024-03-29T04:09:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.378+0000 7f1ff8ccd640 1 --2- 172.21.15.97:0/2913086625 >> v2:172.21.15.154:3300/0 conn(0x7f1ff4071270 0x7f1ff40cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.379+0000 7f1ff94ce640 1 --2- 172.21.15.97:0/2913086625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40cff70 0x7f1ff415e010 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40a0480 tx=0x7f1fec02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3de7ab0d71bd673 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.379+0000 7f1ff37fe640 1 -- 172.21.15.97:0/2913086625 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec02cb20 con 0x7f1ff40cff70 2024-03-29T04:09:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.379+0000 7f1ff37fe640 1 -- 172.21.15.97:0/2913086625 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec02ccc0 con 0x7f1ff40cff70 2024-03-29T04:09:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.379+0000 7f1ff37fe640 1 -- 172.21.15.97:0/2913086625 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec034610 con 0x7f1ff40cff70 2024-03-29T04:09:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.380+0000 7f1ffaf31640 1 -- 172.21.15.97:0/2913086625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40cff70 msgr2=0x7f1ff415e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.380+0000 7f1ffaf31640 1 --2- 172.21.15.97:0/2913086625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40cff70 0x7f1ff415e010 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40a0480 tx=0x7f1fec02bf80 comp rx=0 tx=0).stop 2024-03-29T04:09:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.380+0000 7f1ffaf31640 1 -- 172.21.15.97:0/2913086625 shutdown_connections 2024-03-29T04:09:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.380+0000 7f1ffaf31640 1 --2- 172.21.15.97:0/2913086625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40cff70 0x7f1ff415e010 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.380+0000 7f1ffaf31640 1 --2- 172.21.15.97:0/2913086625 >> v2:172.21.15.154:3300/0 conn(0x7f1ff4071270 0x7f1ff40cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.380+0000 7f1ffaf31640 1 -- 172.21.15.97:0/2913086625 >> 172.21.15.97:0/2913086625 conn(0x7f1ff40b3e90 msgr2=0x7f1ff40b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.380+0000 7f1ffaf31640 1 -- 172.21.15.97:0/2913086625 shutdown_connections 2024-03-29T04:09:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.380+0000 7f1ffaf31640 1 -- 172.21.15.97:0/2913086625 wait complete. 2024-03-29T04:09:41.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.381+0000 7f1ffaf31640 1 Processor -- start 2024-03-29T04:09:41.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.381+0000 7f1ffaf31640 1 -- start start 2024-03-29T04:09:41.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ffaf31640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ff4071270 0x7f1ff415fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ffaf31640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff4071650 0x7f1ff40b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ffaf31640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40b7870 0x7f1ff40c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ffaf31640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff40ce1d0 con 0x7f1ff40b7870 2024-03-29T04:09:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ffaf31640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff40d0ca0 con 0x7f1ff4071650 2024-03-29T04:09:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ffaf31640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff40ce350 con 0x7f1ff4071270 2024-03-29T04:09:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ff94ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40b7870 0x7f1ff40c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ff94ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40b7870 0x7f1ff40c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39430/0 (socket says 172.21.15.97:39430) 2024-03-29T04:09:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ff94ce640 1 -- 172.21.15.97:0/615667299 learned_addr learned my addr 172.21.15.97:0/615667299 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ff3fff640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff4071650 0x7f1ff40b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ff8ccd640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ff4071270 0x7f1ff415fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ff94ce640 1 -- 172.21.15.97:0/615667299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ff4071270 msgr2=0x7f1ff415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ff94ce640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ff4071270 0x7f1ff415fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ff94ce640 1 -- 172.21.15.97:0/615667299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff4071650 msgr2=0x7f1ff40b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ff94ce640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff4071650 0x7f1ff40b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ff94ce640 1 -- 172.21.15.97:0/615667299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff40b7d60 con 0x7f1ff40b7870 2024-03-29T04:09:41.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.382+0000 7f1ff94ce640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40b7870 0x7f1ff40c8640 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009f40 tx=0x7f1fec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.383+0000 7f1ff17fa640 1 -- 172.21.15.97:0/615667299 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec043020 con 0x7f1ff40b7870 2024-03-29T04:09:41.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.383+0000 7f1ff17fa640 1 -- 172.21.15.97:0/615667299 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec03e450 con 0x7f1ff40b7870 2024-03-29T04:09:41.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.383+0000 7f1ff17fa640 1 -- 172.21.15.97:0/615667299 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec009510 con 0x7f1ff40b7870 2024-03-29T04:09:41.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.383+0000 7f1ffaf31640 1 -- 172.21.15.97:0/615667299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff41fc0a0 con 0x7f1ff40b7870 2024-03-29T04:09:41.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.383+0000 7f1ffaf31640 1 -- 172.21.15.97:0/615667299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff41fc660 con 0x7f1ff40b7870 2024-03-29T04:09:41.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.384+0000 7f1ffaf31640 1 -- 172.21.15.97:0/615667299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff4068640 con 0x7f1ff40b7870 2024-03-29T04:09:41.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.384+0000 7f1ff17fa640 1 -- 172.21.15.97:0/615667299 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1fec046020 con 0x7f1ff40b7870 2024-03-29T04:09:41.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.384+0000 7f1ff17fa640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1fd806c530 0x7f1fd806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.384+0000 7f1ff17fa640 1 -- 172.21.15.97:0/615667299 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f1fec0ada20 con 0x7f1ff40b7870 2024-03-29T04:09:41.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.384+0000 7f1ff8ccd640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1fd806c530 0x7f1fd806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.385+0000 7f1ff8ccd640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1fd806c530 0x7f1fd806e9f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1ff415e3b0 tx=0x7f1fe8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.389+0000 7f1ff17fa640 1 -- 172.21.15.97:0/615667299 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1fec04d030 con 0x7f1ff40b7870 2024-03-29T04:09:41.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.562+0000 7f1ffaf31640 1 -- 172.21.15.97:0/615667299 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f1ff40b8040 con 0x7f1fd806c530 2024-03-29T04:09:41.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.562+0000 7f1ff17fa640 1 -- 172.21.15.97:0/615667299 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1ff40b8040 con 0x7f1fd806c530 2024-03-29T04:09:41.542 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 3 to deep-scrub 2024-03-29T04:09:41.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.564+0000 7f1ffaf31640 1 -- 172.21.15.97:0/615667299 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1fd806c530 msgr2=0x7f1fd806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.564+0000 7f1ffaf31640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1fd806c530 0x7f1fd806e9f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1ff415e3b0 tx=0x7f1fe8008040 comp rx=0 tx=0).stop 2024-03-29T04:09:41.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.564+0000 7f1ffaf31640 1 -- 172.21.15.97:0/615667299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40b7870 msgr2=0x7f1ff40c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.564+0000 7f1ffaf31640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40b7870 0x7f1ff40c8640 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009f40 tx=0x7f1fec004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:41.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.564+0000 7f1ffaf31640 1 -- 172.21.15.97:0/615667299 shutdown_connections 2024-03-29T04:09:41.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.564+0000 7f1ffaf31640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f1fd806c530 0x7f1fd806e9f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.564+0000 7f1ffaf31640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40b7870 0x7f1ff40c8640 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.564+0000 7f1ffaf31640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff4071650 0x7f1ff40b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.564+0000 7f1ffaf31640 1 --2- 172.21.15.97:0/615667299 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ff4071270 0x7f1ff415fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.564+0000 7f1ffaf31640 1 -- 172.21.15.97:0/615667299 >> 172.21.15.97:0/615667299 conn(0x7f1ff40b3e90 msgr2=0x7f1ff4160890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.564+0000 7f1ffaf31640 1 -- 172.21.15.97:0/615667299 shutdown_connections 2024-03-29T04:09:41.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.564+0000 7f1ffaf31640 1 -- 172.21.15.97:0/615667299 wait complete. 2024-03-29T04:09:41.554 INFO:tasks.ceph:Scrubbing osd.4 2024-03-29T04:09:41.555 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:09:41.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.720+0000 7f43215a8640 1 Processor -- start 2024-03-29T04:09:41.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.720+0000 7f43215a8640 1 -- start start 2024-03-29T04:09:41.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f43215a8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f431c15e780 0x7f431c162ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f43215a8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f431c15eb60 0x7f431c0d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f43215a8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f431c0b37c0 con 0x7f431c0d4f20 2024-03-29T04:09:41.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f43215a8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f431c163430 con 0x7f431c15eb60 2024-03-29T04:09:41.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f43215a8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f431c0cfde0 con 0x7f431c15e780 2024-03-29T04:09:41.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f431b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f431c15eb60 0x7f431c0d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f43127fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f431c0d4f20 0x7f431c0b1170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41614/0 (socket says 172.21.15.97:41614) 2024-03-29T04:09:41.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f43127fc640 1 -- 172.21.15.97:0/1778971275 learned_addr learned my addr 172.21.15.97:0/1778971275 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:41.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f431b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f431c15eb60 0x7f431c0d0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39444/0 (socket says 172.21.15.97:39444) 2024-03-29T04:09:41.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f431b7fe640 1 -- 172.21.15.97:0/1778971275 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f431c15e780 msgr2=0x7f431c162ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:41.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f431affd640 1 --2- 172.21.15.97:0/1778971275 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f431c15e780 0x7f431c162ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f431b7fe640 1 --2- 172.21.15.97:0/1778971275 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f431c15e780 0x7f431c162ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f431b7fe640 1 -- 172.21.15.97:0/1778971275 >> v1:172.21.15.97:6789/0 conn(0x7f431c0d4f20 legacy=0x7f431c0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f431a7fc640 1 -- 172.21.15.97:0/1778971275 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 407043251 0 0) 0x7f431c0b37c0 con 0x7f431c0d4f20 2024-03-29T04:09:41.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.721+0000 7f431b7fe640 1 -- 172.21.15.97:0/1778971275 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c0d53e0 con 0x7f431c15eb60 2024-03-29T04:09:41.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.722+0000 7f431b7fe640 1 --2- 172.21.15.97:0/1778971275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c15eb60 0x7f431c0d0270 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4314009d90 tx=0x7f431402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c1c21312543492 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.722+0000 7f431a7fc640 1 -- 172.21.15.97:0/1778971275 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314039070 con 0x7f431c15eb60 2024-03-29T04:09:41.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.722+0000 7f431a7fc640 1 -- 172.21.15.97:0/1778971275 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f431402cc50 con 0x7f431c15eb60 2024-03-29T04:09:41.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.722+0000 7f431a7fc640 1 -- 172.21.15.97:0/1778971275 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314034440 con 0x7f431c15eb60 2024-03-29T04:09:41.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.723+0000 7f43215a8640 1 -- 172.21.15.97:0/1778971275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c15eb60 msgr2=0x7f431c0d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.723+0000 7f43215a8640 1 --2- 172.21.15.97:0/1778971275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c15eb60 0x7f431c0d0270 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4314009d90 tx=0x7f431402bea0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.723+0000 7f43215a8640 1 -- 172.21.15.97:0/1778971275 shutdown_connections 2024-03-29T04:09:41.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.723+0000 7f43215a8640 1 --2- 172.21.15.97:0/1778971275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c15eb60 0x7f431c0d0270 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.723+0000 7f43215a8640 1 --2- 172.21.15.97:0/1778971275 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f431c15e780 0x7f431c162ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.723+0000 7f43215a8640 1 -- 172.21.15.97:0/1778971275 >> 172.21.15.97:0/1778971275 conn(0x7f431c08bfc0 msgr2=0x7f431c0d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.723+0000 7f43215a8640 1 -- 172.21.15.97:0/1778971275 shutdown_connections 2024-03-29T04:09:41.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.723+0000 7f43215a8640 1 -- 172.21.15.97:0/1778971275 wait complete. 2024-03-29T04:09:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.724+0000 7f43215a8640 1 Processor -- start 2024-03-29T04:09:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.724+0000 7f43215a8640 1 -- start start 2024-03-29T04:09:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f43215a8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f431c0d4f20 0x7f431c158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f43215a8640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f431c15e780 0x7f431c1f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f43215a8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c15eb60 0x7f431c0ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f43215a8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f431c0cf440 con 0x7f431c15eb60 2024-03-29T04:09:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f43215a8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f431c0cfa70 con 0x7f431c0d4f20 2024-03-29T04:09:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f43215a8640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f431c0b1880 con 0x7f431c15e780 2024-03-29T04:09:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f431b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c15eb60 0x7f431c0ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f431b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c15eb60 0x7f431c0ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39454/0 (socket says 172.21.15.97:39454) 2024-03-29T04:09:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f431b7fe640 1 -- 172.21.15.97:0/2488102592 learned_addr learned my addr 172.21.15.97:0/2488102592 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f43127fc640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f431c15e780 0x7f431c1f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f431affd640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f431c0d4f20 0x7f431c158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f431b7fe640 1 -- 172.21.15.97:0/2488102592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f431c15e780 msgr2=0x7f431c1f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f431b7fe640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f431c15e780 0x7f431c1f7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f431b7fe640 1 -- 172.21.15.97:0/2488102592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f431c0d4f20 msgr2=0x7f431c158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f431b7fe640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f431c0d4f20 0x7f431c158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f431b7fe640 1 -- 172.21.15.97:0/2488102592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c1f7e90 con 0x7f431c15eb60 2024-03-29T04:09:41.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f43127fc640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f431c15e780 0x7f431c1f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:41.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f431affd640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f431c0d4f20 0x7f431c158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:41.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.725+0000 7f431b7fe640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c15eb60 0x7f431c0ce3c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4314009eb0 tx=0x7f4314004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.726+0000 7f4313fff640 1 -- 172.21.15.97:0/2488102592 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314039050 con 0x7f431c15eb60 2024-03-29T04:09:41.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.726+0000 7f4313fff640 1 -- 172.21.15.97:0/2488102592 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4314045070 con 0x7f431c15eb60 2024-03-29T04:09:41.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.726+0000 7f4313fff640 1 -- 172.21.15.97:0/2488102592 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431403e610 con 0x7f431c15eb60 2024-03-29T04:09:41.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.726+0000 7f43215a8640 1 -- 172.21.15.97:0/2488102592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c1f8170 con 0x7f431c15eb60 2024-03-29T04:09:41.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.726+0000 7f43215a8640 1 -- 172.21.15.97:0/2488102592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c160030 con 0x7f431c15eb60 2024-03-29T04:09:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.726+0000 7f43215a8640 1 -- 172.21.15.97:0/2488102592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f431c1f8690 con 0x7f431c15eb60 2024-03-29T04:09:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.727+0000 7f4313fff640 1 -- 172.21.15.97:0/2488102592 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f431403e7b0 con 0x7f431c15eb60 2024-03-29T04:09:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.727+0000 7f4313fff640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f42fc06c530 0x7f42fc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.728+0000 7f4313fff640 1 -- 172.21.15.97:0/2488102592 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f43140adfb0 con 0x7f431c15eb60 2024-03-29T04:09:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.728+0000 7f431affd640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f42fc06c530 0x7f42fc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.728+0000 7f4313fff640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] conn(0x7f42fc0723f0 0x7f42fc074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.728+0000 7f4313fff640 1 -- 172.21.15.97:0/2488102592 --> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4314036e90 con 0x7f42fc0723f0 2024-03-29T04:09:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.728+0000 7f4313fff640 1 -- 172.21.15.97:0/2488102592 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f43140b5050 con 0x7f431c15eb60 2024-03-29T04:09:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.728+0000 7f43127fc640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] conn(0x7f42fc0723f0 0x7f42fc074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.728+0000 7f431affd640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f42fc06c530 0x7f42fc06e9f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f430c008890 tx=0x7f430c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.728+0000 7f43127fc640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] conn(0x7f42fc0723f0 0x7f42fc074810 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.729+0000 7f4313fff640 1 -- 172.21.15.97:0/2488102592 <== osd.4 v2:172.21.15.154:6824/2981976178 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4314036e90 con 0x7f42fc0723f0 2024-03-29T04:09:41.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.742+0000 7f43215a8640 1 -- 172.21.15.97:0/2488102592 --> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f431c068640 con 0x7f42fc0723f0 2024-03-29T04:09:41.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.743+0000 7f4313fff640 1 -- 172.21.15.97:0/2488102592 <== osd.4 v2:172.21.15.154:6824/2981976178 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f431c068640 con 0x7f42fc0723f0 2024-03-29T04:09:41.722 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:09:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:09:41.723 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:09:41.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.743+0000 7f43215a8640 1 -- 172.21.15.97:0/2488102592 >> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] conn(0x7f42fc0723f0 msgr2=0x7f42fc074810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.743+0000 7f43215a8640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.154:6824/2981976178,v1:172.21.15.154:6825/2981976178] conn(0x7f42fc0723f0 0x7f42fc074810 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.743+0000 7f43215a8640 1 -- 172.21.15.97:0/2488102592 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f42fc06c530 msgr2=0x7f42fc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.743+0000 7f43215a8640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f42fc06c530 0x7f42fc06e9f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f430c008890 tx=0x7f430c008040 comp rx=0 tx=0).stop 2024-03-29T04:09:41.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.743+0000 7f43215a8640 1 -- 172.21.15.97:0/2488102592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c15eb60 msgr2=0x7f431c0ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.743+0000 7f43215a8640 1 --2- 172.21.15.97:0/2488102592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c15eb60 0x7f431c0ce3c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4314009eb0 tx=0x7f4314004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:41.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.743+0000 7f431b7fe640 1 -- 172.21.15.97:0/2488102592 reap_dead start 2024-03-29T04:09:41.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.743+0000 7f43215a8640 1 -- 172.21.15.97:0/2488102592 shutdown_connections 2024-03-29T04:09:41.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.743+0000 7f43215a8640 1 -- 172.21.15.97:0/2488102592 >> 172.21.15.97:0/2488102592 conn(0x7f431c08bfc0 msgr2=0x7f431c06c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.743+0000 7f43215a8640 1 -- 172.21.15.97:0/2488102592 shutdown_connections 2024-03-29T04:09:41.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.743+0000 7f43215a8640 1 -- 172.21.15.97:0/2488102592 wait complete. 2024-03-29T04:09:41.733 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-29T04:09:41.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.856+0000 7f008e4d9640 1 Processor -- start 2024-03-29T04:09:41.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.856+0000 7f008e4d9640 1 -- start start 2024-03-29T04:09:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.856+0000 7f008e4d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00880b1ca0 0x7f00880b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f008e4d9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00880b2100 con 0x7f00880b28a0 2024-03-29T04:09:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f008e4d9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00880b0ee0 con 0x7f00880d05c0 2024-03-29T04:09:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f008e4d9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f00880b11c0 con 0x7f00880b1ca0 2024-03-29T04:09:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f008ca76640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f00880b28a0 0x7f00880cb8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41624/0 (socket says 172.21.15.97:41624) 2024-03-29T04:09:41.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f008ca76640 1 -- 172.21.15.97:0/2471043776 learned_addr learned my addr 172.21.15.97:0/2471043776 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f00877fe640 1 --2- 172.21.15.97:0/2471043776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00880b1ca0 0x7f00880b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 233770664 0 0) 0x7f0080001490 con 0x7f00880b28a0 2024-03-29T04:09:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0070003520 con 0x7f00880b28a0 2024-03-29T04:09:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4236879062 0 0) 0x7f0070003520 con 0x7f00880b28a0 2024-03-29T04:09:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00880b1ca0 msgr2=0x7f00880b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 --2- 172.21.15.97:0/2471043776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00880b1ca0 0x7f00880b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 >> v1:172.21.15.154:6789/0 conn(0x7f00880d05c0 legacy=0x7f00880d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00880b14a0 con 0x7f00880b28a0 2024-03-29T04:09:41.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f0080001bc0 con 0x7f00880b28a0 2024-03-29T04:09:41.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 >> v1:172.21.15.97:6789/0 conn(0x7f00880b28a0 legacy=0x7f00880cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 --2- 172.21.15.97:0/2471043776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00700039e0 0x7f0070003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 --2- 172.21.15.97:0/2471043776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00880b1ca0 0x7f0070004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 --2- 172.21.15.97:0/2471043776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0070004d40 0x7f00700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f00880b11c0 con 0x7f00700039e0 2024-03-29T04:09:41.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f007000b9c0 con 0x7f0070004d40 2024-03-29T04:09:41.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f007000bbc0 con 0x7f00880b1ca0 2024-03-29T04:09:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.857+0000 7f0087fff640 1 --2- 172.21.15.97:0/2471043776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00700039e0 0x7f0070003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.858+0000 7f00877fe640 1 --2- 172.21.15.97:0/2471043776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00880b1ca0 0x7f0070004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.858+0000 7f008ca76640 1 --2- 172.21.15.97:0/2471043776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0070004d40 0x7f00700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.858+0000 7f0087fff640 1 -- 172.21.15.97:0/2471043776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00880b1ca0 msgr2=0x7f0070004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.858+0000 7f0087fff640 1 --2- 172.21.15.97:0/2471043776 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00880b1ca0 0x7f0070004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.858+0000 7f0087fff640 1 -- 172.21.15.97:0/2471043776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0070004d40 msgr2=0x7f00700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.858+0000 7f0087fff640 1 --2- 172.21.15.97:0/2471043776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0070004d40 0x7f00700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.858+0000 7f0087fff640 1 -- 172.21.15.97:0/2471043776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0070003520 con 0x7f00700039e0 2024-03-29T04:09:41.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.858+0000 7f0087fff640 1 --2- 172.21.15.97:0/2471043776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00700039e0 0x7f0070003dc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0078001860 tx=0x7f0070005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935a069629670c0c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.858+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078038070 con 0x7f00700039e0 2024-03-29T04:09:41.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.858+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0078033660 con 0x7f00700039e0 2024-03-29T04:09:41.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.858+0000 7f0086ffd640 1 -- 172.21.15.97:0/2471043776 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078001430 con 0x7f00700039e0 2024-03-29T04:09:41.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.859+0000 7f008e4d9640 1 -- 172.21.15.97:0/2471043776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00700039e0 msgr2=0x7f0070003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.859+0000 7f008e4d9640 1 --2- 172.21.15.97:0/2471043776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00700039e0 0x7f0070003dc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0078001860 tx=0x7f0070005830 comp rx=0 tx=0).stop 2024-03-29T04:09:41.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.859+0000 7f008ca76640 1 -- 172.21.15.97:0/2471043776 reap_dead start 2024-03-29T04:09:41.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.859+0000 7f008e4d9640 1 -- 172.21.15.97:0/2471043776 shutdown_connections 2024-03-29T04:09:41.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.859+0000 7f008e4d9640 1 -- 172.21.15.97:0/2471043776 >> 172.21.15.97:0/2471043776 conn(0x7f00880b2e60 msgr2=0x7f00880b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.859+0000 7f008e4d9640 1 -- 172.21.15.97:0/2471043776 shutdown_connections 2024-03-29T04:09:41.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.859+0000 7f008e4d9640 1 -- 172.21.15.97:0/2471043776 wait complete. 2024-03-29T04:09:41.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.860+0000 7f008e4d9640 1 Processor -- start 2024-03-29T04:09:41.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.860+0000 7f008e4d9640 1 -- start start 2024-03-29T04:09:41.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f008e4d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00700039e0 0x7f00880cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f008e4d9640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00881f7970 0x7f00880cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f008e4d9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00881f7df0 0x7f00880cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f008e4d9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f00880ceae0 con 0x7f00700039e0 2024-03-29T04:09:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f008e4d9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f00880ce960 con 0x7f00881f7df0 2024-03-29T04:09:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f008e4d9640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f008815c940 con 0x7f00881f7970 2024-03-29T04:09:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f0087fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00700039e0 0x7f00880cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f0087fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00700039e0 0x7f00880cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39468/0 (socket says 172.21.15.97:39468) 2024-03-29T04:09:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f0087fff640 1 -- 172.21.15.97:0/452803307 learned_addr learned my addr 172.21.15.97:0/452803307 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f008ca76640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00881f7df0 0x7f00880cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f0087fff640 1 -- 172.21.15.97:0/452803307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00881f7970 msgr2=0x7f00880cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f00877fe640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00881f7970 0x7f00880cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f0087fff640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00881f7970 0x7f00880cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f0087fff640 1 -- 172.21.15.97:0/452803307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00881f7df0 msgr2=0x7f00880cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f0087fff640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00881f7df0 0x7f00880cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.861+0000 7f0087fff640 1 -- 172.21.15.97:0/452803307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00881f83c0 con 0x7f00700039e0 2024-03-29T04:09:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.862+0000 7f0087fff640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00700039e0 0x7f00880cb1d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f00780018c0 tx=0x7f0078001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.862+0000 7f0084ff9640 1 -- 172.21.15.97:0/452803307 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078041020 con 0x7f00700039e0 2024-03-29T04:09:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.862+0000 7f0084ff9640 1 -- 172.21.15.97:0/452803307 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0078005270 con 0x7f00700039e0 2024-03-29T04:09:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.862+0000 7f008e4d9640 1 -- 172.21.15.97:0/452803307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00881f86a0 con 0x7f00700039e0 2024-03-29T04:09:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.862+0000 7f0084ff9640 1 -- 172.21.15.97:0/452803307 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078038040 con 0x7f00700039e0 2024-03-29T04:09:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.862+0000 7f008e4d9640 1 -- 172.21.15.97:0/452803307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0088157350 con 0x7f00700039e0 2024-03-29T04:09:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.863+0000 7f008e4d9640 1 -- 172.21.15.97:0/452803307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0088068d50 con 0x7f00700039e0 2024-03-29T04:09:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.863+0000 7f0084ff9640 1 -- 172.21.15.97:0/452803307 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0078044020 con 0x7f00700039e0 2024-03-29T04:09:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.864+0000 7f0084ff9640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f007406c530 0x7f007406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.864+0000 7f0084ff9640 1 -- 172.21.15.97:0/452803307 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f00780ac800 con 0x7f00700039e0 2024-03-29T04:09:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.864+0000 7f00877fe640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f007406c530 0x7f007406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.864+0000 7f00877fe640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f007406c530 0x7f007406e9f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0088157170 tx=0x7f007c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:41.868+0000 7f0084ff9640 1 -- 172.21.15.97:0/452803307 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f007804b030 con 0x7f00700039e0 2024-03-29T04:09:42.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.042+0000 7f008e4d9640 1 -- 172.21.15.97:0/452803307 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f00880cc430 con 0x7f007406c530 2024-03-29T04:09:42.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.043+0000 7f0084ff9640 1 -- 172.21.15.97:0/452803307 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f00880cc430 con 0x7f007406c530 2024-03-29T04:09:42.023 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 4 to deep-scrub 2024-03-29T04:09:42.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.045+0000 7f008e4d9640 1 -- 172.21.15.97:0/452803307 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f007406c530 msgr2=0x7f007406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.045+0000 7f008e4d9640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f007406c530 0x7f007406e9f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0088157170 tx=0x7f007c008040 comp rx=0 tx=0).stop 2024-03-29T04:09:42.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.045+0000 7f008e4d9640 1 -- 172.21.15.97:0/452803307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00700039e0 msgr2=0x7f00880cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.045+0000 7f008e4d9640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00700039e0 0x7f00880cb1d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f00780018c0 tx=0x7f0078001230 comp rx=0 tx=0).stop 2024-03-29T04:09:42.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.045+0000 7f008e4d9640 1 -- 172.21.15.97:0/452803307 shutdown_connections 2024-03-29T04:09:42.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.045+0000 7f008e4d9640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f007406c530 0x7f007406e9f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.045+0000 7f008e4d9640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00881f7df0 0x7f00880cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.045+0000 7f008e4d9640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00881f7970 0x7f00880cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.045+0000 7f008e4d9640 1 --2- 172.21.15.97:0/452803307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00700039e0 0x7f00880cb1d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.045+0000 7f008e4d9640 1 -- 172.21.15.97:0/452803307 >> 172.21.15.97:0/452803307 conn(0x7f00880b2e60 msgr2=0x7f00880749e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.045+0000 7f008e4d9640 1 -- 172.21.15.97:0/452803307 shutdown_connections 2024-03-29T04:09:42.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.045+0000 7f008e4d9640 1 -- 172.21.15.97:0/452803307 wait complete. 2024-03-29T04:09:42.035 INFO:tasks.ceph:Scrubbing osd.5 2024-03-29T04:09:42.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:09:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.160+0000 7f833804e640 1 Processor -- start 2024-03-29T04:09:42.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.160+0000 7f833804e640 1 -- start start 2024-03-29T04:09:42.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.160+0000 7f833804e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833015d630 0x7f83300d0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f833804e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f833006e300 con 0x7f8330156ba0 2024-03-29T04:09:42.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f833804e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f833006e5e0 con 0x7f83300d09d0 2024-03-29T04:09:42.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f833804e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f833015ce50 con 0x7f833015d630 2024-03-29T04:09:42.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8335dea640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8330156ba0 0x7f833006d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41626/0 (socket says 172.21.15.97:41626) 2024-03-29T04:09:42.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8335dea640 1 -- 172.21.15.97:0/3962963545 learned_addr learned my addr 172.21.15.97:0/3962963545 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:42.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2834902360 0 0) 0x7f833006e300 con 0x7f8330156ba0 2024-03-29T04:09:42.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f831c003520 con 0x7f8330156ba0 2024-03-29T04:09:42.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f83355e9640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833015d630 0x7f83300d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2594118847 0 0) 0x7f831c003520 con 0x7f8330156ba0 2024-03-29T04:09:42.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833015d630 msgr2=0x7f83300d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:42.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833015d630 0x7f83300d0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 >> v1:172.21.15.154:6789/0 conn(0x7f83300d09d0 legacy=0x7f8330160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833015d130 con 0x7f8330156ba0 2024-03-29T04:09:42.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f8320001670 con 0x7f8330156ba0 2024-03-29T04:09:42.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 >> v1:172.21.15.97:6789/0 conn(0x7f8330156ba0 legacy=0x7f833006d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f831c0039e0 0x7f831c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833015d630 0x7f831c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f831c004d40 0x7f831c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f833015ce50 con 0x7f831c004d40 2024-03-29T04:09:42.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f831c00bb20 con 0x7f831c0039e0 2024-03-29T04:09:42.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f831c00bd20 con 0x7f833015d630 2024-03-29T04:09:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f83365eb640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f831c004d40 0x7f831c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f8335dea640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f831c0039e0 0x7f831c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.161+0000 7f83355e9640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833015d630 0x7f831c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.162+0000 7f83365eb640 1 -- 172.21.15.97:0/3962963545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833015d630 msgr2=0x7f831c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.162+0000 7f83365eb640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833015d630 0x7f831c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.162+0000 7f83365eb640 1 -- 172.21.15.97:0/3962963545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f831c0039e0 msgr2=0x7f831c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.162+0000 7f83365eb640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f831c0039e0 0x7f831c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.162+0000 7f83365eb640 1 -- 172.21.15.97:0/3962963545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c003520 con 0x7f831c004d40 2024-03-29T04:09:42.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.162+0000 7f8335dea640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f831c0039e0 0x7f831c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:42.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.162+0000 7f83355e9640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833015d630 0x7f831c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:42.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.163+0000 7f83365eb640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f831c004d40 0x7f831c0051c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8328001e50 tx=0x7f831c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d68f73576d99ed9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.163+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328001a70 con 0x7f831c004d40 2024-03-29T04:09:42.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.163+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8328003b70 con 0x7f831c004d40 2024-03-29T04:09:42.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.163+0000 7f8334de8640 1 -- 172.21.15.97:0/3962963545 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83280050d0 con 0x7f831c004d40 2024-03-29T04:09:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.163+0000 7f833804e640 1 -- 172.21.15.97:0/3962963545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f831c004d40 msgr2=0x7f831c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.163+0000 7f833804e640 1 --2- 172.21.15.97:0/3962963545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f831c004d40 0x7f831c0051c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8328001e50 tx=0x7f831c005830 comp rx=0 tx=0).stop 2024-03-29T04:09:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.163+0000 7f83365eb640 1 -- 172.21.15.97:0/3962963545 reap_dead start 2024-03-29T04:09:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.163+0000 7f833804e640 1 -- 172.21.15.97:0/3962963545 shutdown_connections 2024-03-29T04:09:42.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.163+0000 7f833804e640 1 -- 172.21.15.97:0/3962963545 >> 172.21.15.97:0/3962963545 conn(0x7f833006b6f0 msgr2=0x7f833006bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.163+0000 7f833804e640 1 -- 172.21.15.97:0/3962963545 shutdown_connections 2024-03-29T04:09:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.163+0000 7f833804e640 1 -- 172.21.15.97:0/3962963545 wait complete. 2024-03-29T04:09:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.164+0000 7f833804e640 1 Processor -- start 2024-03-29T04:09:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.164+0000 7f833804e640 1 -- start start 2024-03-29T04:09:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.165+0000 7f833804e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833014b1d0 0x7f83301513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.165+0000 7f833804e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f833014b5b0 0x7f8330151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.165+0000 7f833804e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8330099090 0x7f8330099510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.165+0000 7f833804e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f83300ad3d0 con 0x7f8330099090 2024-03-29T04:09:42.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.165+0000 7f833804e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f833015dac0 con 0x7f833014b5b0 2024-03-29T04:09:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.165+0000 7f833804e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f8330156f80 con 0x7f833014b1d0 2024-03-29T04:09:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.165+0000 7f83365eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8330099090 0x7f8330099510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.165+0000 7f83365eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8330099090 0x7f8330099510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39484/0 (socket says 172.21.15.97:39484) 2024-03-29T04:09:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.165+0000 7f83365eb640 1 -- 172.21.15.97:0/2576190531 learned_addr learned my addr 172.21.15.97:0/2576190531 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.165+0000 7f8335dea640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833014b1d0 0x7f83301513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.165+0000 7f83365eb640 1 -- 172.21.15.97:0/2576190531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833014b1d0 msgr2=0x7f83301513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.165+0000 7f83355e9640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f833014b5b0 0x7f8330151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.166+0000 7f83365eb640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833014b1d0 0x7f83301513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.166+0000 7f83365eb640 1 -- 172.21.15.97:0/2576190531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f833014b5b0 msgr2=0x7f8330151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.166+0000 7f83365eb640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f833014b5b0 0x7f8330151900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.166+0000 7f83365eb640 1 -- 172.21.15.97:0/2576190531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8330151f80 con 0x7f8330099090 2024-03-29T04:09:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.166+0000 7f8335dea640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f833014b1d0 0x7f83301513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.166+0000 7f83355e9640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f833014b5b0 0x7f8330151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.166+0000 7f83365eb640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8330099090 0x7f8330099510 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f832802c060 tx=0x7f83280014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.166+0000 7f83267fc640 1 -- 172.21.15.97:0/2576190531 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328007db0 con 0x7f8330099090 2024-03-29T04:09:42.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.166+0000 7f83267fc640 1 -- 172.21.15.97:0/2576190531 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8328003970 con 0x7f8330099090 2024-03-29T04:09:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.166+0000 7f83267fc640 1 -- 172.21.15.97:0/2576190531 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832802f6a0 con 0x7f8330099090 2024-03-29T04:09:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.166+0000 7f833804e640 1 -- 172.21.15.97:0/2576190531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8330152260 con 0x7f8330099090 2024-03-29T04:09:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.166+0000 7f833804e640 1 -- 172.21.15.97:0/2576190531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8330140c20 con 0x7f8330099090 2024-03-29T04:09:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.167+0000 7f833804e640 1 -- 172.21.15.97:0/2576190531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f833014cba0 con 0x7f8330099090 2024-03-29T04:09:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.167+0000 7f83267fc640 1 -- 172.21.15.97:0/2576190531 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8328041030 con 0x7f8330099090 2024-03-29T04:09:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.168+0000 7f83267fc640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f831406c530 0x7f831406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.168+0000 7f83267fc640 1 -- 172.21.15.97:0/2576190531 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f83280ac3a0 con 0x7f8330099090 2024-03-29T04:09:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.168+0000 7f8335dea640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f831406c530 0x7f831406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.168+0000 7f83267fc640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] conn(0x7f83140723f0 0x7f8314074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.168+0000 7f83267fc640 1 -- 172.21.15.97:0/2576190531 --> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8328048e80 con 0x7f83140723f0 2024-03-29T04:09:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.168+0000 7f83267fc640 1 -- 172.21.15.97:0/2576190531 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f83280b3050 con 0x7f8330099090 2024-03-29T04:09:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.168+0000 7f8335dea640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f831406c530 0x7f831406e9f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8320000fb0 tx=0x7f8320001170 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.168+0000 7f83355e9640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] conn(0x7f83140723f0 0x7f8314074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.169+0000 7f83355e9640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] conn(0x7f83140723f0 0x7f8314074810 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.169+0000 7f83267fc640 1 -- 172.21.15.97:0/2576190531 <== osd.5 v2:172.21.15.154:6808/2773817748 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8328048e80 con 0x7f83140723f0 2024-03-29T04:09:42.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.182+0000 7f833804e640 1 -- 172.21.15.97:0/2576190531 --> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f83300aec50 con 0x7f83140723f0 2024-03-29T04:09:42.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.183+0000 7f83267fc640 1 -- 172.21.15.97:0/2576190531 <== osd.5 v2:172.21.15.154:6808/2773817748 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f83300aec50 con 0x7f83140723f0 2024-03-29T04:09:42.162 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:09:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:09:42.163 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:09:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.183+0000 7f833804e640 1 -- 172.21.15.97:0/2576190531 >> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] conn(0x7f83140723f0 msgr2=0x7f8314074810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.183+0000 7f833804e640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.154:6808/2773817748,v1:172.21.15.154:6809/2773817748] conn(0x7f83140723f0 0x7f8314074810 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.183+0000 7f833804e640 1 -- 172.21.15.97:0/2576190531 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f831406c530 msgr2=0x7f831406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.183+0000 7f833804e640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f831406c530 0x7f831406e9f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8320000fb0 tx=0x7f8320001170 comp rx=0 tx=0).stop 2024-03-29T04:09:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.183+0000 7f833804e640 1 -- 172.21.15.97:0/2576190531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8330099090 msgr2=0x7f8330099510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.183+0000 7f833804e640 1 --2- 172.21.15.97:0/2576190531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8330099090 0x7f8330099510 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f832802c060 tx=0x7f83280014a0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.183+0000 7f83365eb640 1 -- 172.21.15.97:0/2576190531 reap_dead start 2024-03-29T04:09:42.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.183+0000 7f833804e640 1 -- 172.21.15.97:0/2576190531 shutdown_connections 2024-03-29T04:09:42.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.183+0000 7f833804e640 1 -- 172.21.15.97:0/2576190531 >> 172.21.15.97:0/2576190531 conn(0x7f833006b6f0 msgr2=0x7f833015dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.184+0000 7f833804e640 1 -- 172.21.15.97:0/2576190531 shutdown_connections 2024-03-29T04:09:42.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.184+0000 7f833804e640 1 -- 172.21.15.97:0/2576190531 wait complete. 2024-03-29T04:09:42.173 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-29T04:09:42.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.294+0000 7f5f9e1fd640 1 Processor -- start 2024-03-29T04:09:42.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.295+0000 7f5f9e1fd640 1 -- start start 2024-03-29T04:09:42.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.295+0000 7f5f9e1fd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5f9815ec10 0x7f5f980c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.295+0000 7f5f9e1fd640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f9815f100 con 0x7f5f980d0730 2024-03-29T04:09:42.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.295+0000 7f5f9e1fd640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f980cebd0 con 0x7f5f980ce740 2024-03-29T04:09:42.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.295+0000 7f5f9e1fd640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f5f9815f370 con 0x7f5f9815ec10 2024-03-29T04:09:42.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.295+0000 7f5f97fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5f980d0730 0x7f5f980b30b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41634/0 (socket says 172.21.15.97:41634) 2024-03-29T04:09:42.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.295+0000 7f5f97fff640 1 -- 172.21.15.97:0/1278708908 learned_addr learned my addr 172.21.15.97:0/1278708908 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:42.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f977fe640 1 --2- 172.21.15.97:0/1278708908 >> v2:172.21.15.154:3300/0 conn(0x7f5f9815ec10 0x7f5f980c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2218181425 0 0) 0x7f5f9815f100 con 0x7f5f980d0730 2024-03-29T04:09:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f80003520 con 0x7f5f980d0730 2024-03-29T04:09:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 990840705 0 0) 0x7f5f80003520 con 0x7f5f980d0730 2024-03-29T04:09:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 >> v2:172.21.15.154:3300/0 conn(0x7f5f9815ec10 msgr2=0x7f5f980c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 --2- 172.21.15.97:0/1278708908 >> v2:172.21.15.154:3300/0 conn(0x7f5f9815ec10 0x7f5f980c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 >> v1:172.21.15.154:6789/0 conn(0x7f5f980ce740 legacy=0x7f5f980d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9815f650 con 0x7f5f980d0730 2024-03-29T04:09:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f5f900019f0 con 0x7f5f980d0730 2024-03-29T04:09:42.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 >> v1:172.21.15.97:6789/0 conn(0x7f5f980d0730 legacy=0x7f5f980b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 --2- 172.21.15.97:0/1278708908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f800039b0 0x7f5f80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 --2- 172.21.15.97:0/1278708908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f80004800 0x7f5f80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 --2- 172.21.15.97:0/1278708908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f800051c0 0x7f5f8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9815f370 con 0x7f5f800039b0 2024-03-29T04:09:42.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8000c160 con 0x7f5f800051c0 2024-03-29T04:09:42.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8000c360 con 0x7f5f80004800 2024-03-29T04:09:42.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f977fe640 1 --2- 172.21.15.97:0/1278708908 >> v2:172.21.15.154:3300/0 conn(0x7f5f9815ec10 0x7f5f980c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:42.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f977fe640 1 --2- 172.21.15.97:0/1278708908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f800039b0 0x7f5f80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f97fff640 1 --2- 172.21.15.97:0/1278708908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f800051c0 0x7f5f8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f96ffd640 1 --2- 172.21.15.97:0/1278708908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f80004800 0x7f5f80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f977fe640 1 -- 172.21.15.97:0/1278708908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f80004800 msgr2=0x7f5f80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:42.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f977fe640 1 --2- 172.21.15.97:0/1278708908 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f80004800 0x7f5f80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f977fe640 1 -- 172.21.15.97:0/1278708908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f800051c0 msgr2=0x7f5f8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:42.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f977fe640 1 --2- 172.21.15.97:0/1278708908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f800051c0 0x7f5f8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f977fe640 1 -- 172.21.15.97:0/1278708908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f80003520 con 0x7f5f800039b0 2024-03-29T04:09:42.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.296+0000 7f5f97fff640 1 -- 172.21.15.97:0/1278708908 reap_dead start 2024-03-29T04:09:42.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.297+0000 7f5f977fe640 1 --2- 172.21.15.97:0/1278708908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f800039b0 0x7f5f80003dd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009c70 tx=0x7f5f80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4500256d85332787 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.297+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88009840 con 0x7f5f800039b0 2024-03-29T04:09:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.297+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f88004020 con 0x7f5f800039b0 2024-03-29T04:09:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.297+0000 7f5f967fc640 1 -- 172.21.15.97:0/1278708908 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88004450 con 0x7f5f800039b0 2024-03-29T04:09:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.298+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/1278708908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f800039b0 msgr2=0x7f5f80003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.298+0000 7f5f9e1fd640 1 --2- 172.21.15.97:0/1278708908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f800039b0 0x7f5f80003dd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009c70 tx=0x7f5f80005830 comp rx=0 tx=0).stop 2024-03-29T04:09:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.298+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/1278708908 shutdown_connections 2024-03-29T04:09:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.298+0000 7f5f9e1fd640 1 --2- 172.21.15.97:0/1278708908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f800039b0 0x7f5f80003dd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.298+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/1278708908 >> 172.21.15.97:0/1278708908 conn(0x7f5f9806b510 msgr2=0x7f5f9806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.298+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/1278708908 shutdown_connections 2024-03-29T04:09:42.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.298+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/1278708908 wait complete. 2024-03-29T04:09:42.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.299+0000 7f5f9e1fd640 1 Processor -- start 2024-03-29T04:09:42.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.299+0000 7f5f9e1fd640 1 -- start start 2024-03-29T04:09:42.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f9e1fd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f9806dc60 0x7f5f980cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f9e1fd640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f980716b0 0x7f5f980ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f9e1fd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f9806e040 0x7f5f9806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f9e1fd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9815fe50 con 0x7f5f9806e040 2024-03-29T04:09:42.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f9e1fd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5f980c6ee0 con 0x7f5f9806dc60 2024-03-29T04:09:42.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f9e1fd640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9815ffd0 con 0x7f5f980716b0 2024-03-29T04:09:42.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f97fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f9806e040 0x7f5f9806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f97fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f9806e040 0x7f5f9806e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39500/0 (socket says 172.21.15.97:39500) 2024-03-29T04:09:42.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f97fff640 1 -- 172.21.15.97:0/4103305086 learned_addr learned my addr 172.21.15.97:0/4103305086 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:42.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f96ffd640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f980716b0 0x7f5f980ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f977fe640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f9806dc60 0x7f5f980cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f97fff640 1 -- 172.21.15.97:0/4103305086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f980716b0 msgr2=0x7f5f980ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f97fff640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f980716b0 0x7f5f980ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f97fff640 1 -- 172.21.15.97:0/4103305086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f9806dc60 msgr2=0x7f5f980cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f97fff640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f9806dc60 0x7f5f980cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f97fff640 1 -- 172.21.15.97:0/4103305086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f98072c10 con 0x7f5f9806e040 2024-03-29T04:09:42.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f977fe640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f9806dc60 0x7f5f980cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:42.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f97fff640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f9806e040 0x7f5f9806e4d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5f90000f80 tx=0x7f5f90007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.300+0000 7f5f7ffff640 1 -- 172.21.15.97:0/4103305086 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9000ab00 con 0x7f5f9806e040 2024-03-29T04:09:42.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.301+0000 7f5f7ffff640 1 -- 172.21.15.97:0/4103305086 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f90019070 con 0x7f5f9806e040 2024-03-29T04:09:42.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.301+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/4103305086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f98071b70 con 0x7f5f9806e040 2024-03-29T04:09:42.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.301+0000 7f5f7ffff640 1 -- 172.21.15.97:0/4103305086 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f900133f0 con 0x7f5f9806e040 2024-03-29T04:09:42.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.301+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/4103305086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f98072130 con 0x7f5f9806e040 2024-03-29T04:09:42.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.302+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/4103305086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f98068640 con 0x7f5f9806e040 2024-03-29T04:09:42.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.302+0000 7f5f7ffff640 1 -- 172.21.15.97:0/4103305086 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5f90013590 con 0x7f5f9806e040 2024-03-29T04:09:42.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.302+0000 7f5f7ffff640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5f8406c530 0x7f5f8406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.302+0000 7f5f7ffff640 1 -- 172.21.15.97:0/4103305086 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f5f9008b960 con 0x7f5f9806e040 2024-03-29T04:09:42.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.302+0000 7f5f977fe640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5f8406c530 0x7f5f8406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.303+0000 7f5f977fe640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5f8406c530 0x7f5f8406e9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5f88035000 tx=0x7f5f88030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.307+0000 7f5f7ffff640 1 -- 172.21.15.97:0/4103305086 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5f9005a280 con 0x7f5f9806e040 2024-03-29T04:09:42.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.483+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/4103305086 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f5f980b4f60 con 0x7f5f8406c530 2024-03-29T04:09:42.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.483+0000 7f5f7ffff640 1 -- 172.21.15.97:0/4103305086 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5f980b4f60 con 0x7f5f8406c530 2024-03-29T04:09:42.463 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 5 to deep-scrub 2024-03-29T04:09:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.485+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/4103305086 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5f8406c530 msgr2=0x7f5f8406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.485+0000 7f5f9e1fd640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5f8406c530 0x7f5f8406e9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5f88035000 tx=0x7f5f88030000 comp rx=0 tx=0).stop 2024-03-29T04:09:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.485+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/4103305086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f9806e040 msgr2=0x7f5f9806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.485+0000 7f5f9e1fd640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f9806e040 0x7f5f9806e4d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5f90000f80 tx=0x7f5f90007b00 comp rx=0 tx=0).stop 2024-03-29T04:09:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.485+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/4103305086 shutdown_connections 2024-03-29T04:09:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.485+0000 7f5f9e1fd640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f5f8406c530 0x7f5f8406e9f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.485+0000 7f5f9e1fd640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f9806e040 0x7f5f9806e4d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.485+0000 7f5f9e1fd640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f980716b0 0x7f5f980ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.485+0000 7f5f9e1fd640 1 --2- 172.21.15.97:0/4103305086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f9806dc60 0x7f5f980cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.485+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/4103305086 >> 172.21.15.97:0/4103305086 conn(0x7f5f9806b510 msgr2=0x7f5f980cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.485+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/4103305086 shutdown_connections 2024-03-29T04:09:42.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.485+0000 7f5f9e1fd640 1 -- 172.21.15.97:0/4103305086 wait complete. 2024-03-29T04:09:42.475 INFO:tasks.ceph:Scrubbing osd.6 2024-03-29T04:09:42.475 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:09:42.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.600+0000 7fed7728f640 1 Processor -- start 2024-03-29T04:09:42.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.600+0000 7fed7728f640 1 -- start start 2024-03-29T04:09:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.601+0000 7fed7728f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed7006b510 0x7fed7006b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.601+0000 7fed7728f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed700b2c10 0x7fed700b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.601+0000 7fed7728f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed700ce810 con 0x7fed700d1100 2024-03-29T04:09:42.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.601+0000 7fed7728f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fed7006be30 con 0x7fed700b2c10 2024-03-29T04:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.601+0000 7fed7728f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fed700b6420 con 0x7fed7006b510 2024-03-29T04:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.601+0000 7fed7482a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed700b2c10 0x7fed700b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.601+0000 7fed7582c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fed700d1100 0x7fed700d1590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41640/0 (socket says 172.21.15.97:41640) 2024-03-29T04:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.601+0000 7fed7582c640 1 -- 172.21.15.97:0/3608336767 learned_addr learned my addr 172.21.15.97:0/3608336767 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.601+0000 7fed7502b640 1 --2- 172.21.15.97:0/3608336767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed7006b510 0x7fed7006b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.601+0000 7fed7482a640 1 -- 172.21.15.97:0/3608336767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed7006b510 msgr2=0x7fed7006b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.601+0000 7fed7482a640 1 --2- 172.21.15.97:0/3608336767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed7006b510 0x7fed7006b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.601+0000 7fed7482a640 1 -- 172.21.15.97:0/3608336767 >> v1:172.21.15.97:6789/0 conn(0x7fed700d1100 legacy=0x7fed700d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.602+0000 7fed67fff640 1 -- 172.21.15.97:0/3608336767 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 117661152 0 0) 0x7fed700ce810 con 0x7fed700d1100 2024-03-29T04:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.602+0000 7fed7482a640 1 -- 172.21.15.97:0/3608336767 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed700b6700 con 0x7fed700b2c10 2024-03-29T04:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.602+0000 7fed7502b640 1 --2- 172.21.15.97:0/3608336767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed7006b510 0x7fed7006b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:42.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.602+0000 7fed7482a640 1 --2- 172.21.15.97:0/3608336767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed700b2c10 0x7fed700b43b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fed600097a0 tx=0x7fed6002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7670ef27870819a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.602+0000 7fed67fff640 1 -- 172.21.15.97:0/3608336767 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed600315a0 con 0x7fed700b2c10 2024-03-29T04:09:42.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.602+0000 7fed67fff640 1 -- 172.21.15.97:0/3608336767 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed60037070 con 0x7fed700b2c10 2024-03-29T04:09:42.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.602+0000 7fed67fff640 1 -- 172.21.15.97:0/3608336767 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60031960 con 0x7fed700b2c10 2024-03-29T04:09:42.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.603+0000 7fed7728f640 1 -- 172.21.15.97:0/3608336767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed700b2c10 msgr2=0x7fed700b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.603+0000 7fed7728f640 1 --2- 172.21.15.97:0/3608336767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed700b2c10 0x7fed700b43b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fed600097a0 tx=0x7fed6002b840 comp rx=0 tx=0).stop 2024-03-29T04:09:42.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.603+0000 7fed7728f640 1 -- 172.21.15.97:0/3608336767 shutdown_connections 2024-03-29T04:09:42.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.603+0000 7fed7728f640 1 --2- 172.21.15.97:0/3608336767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed700b2c10 0x7fed700b43b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.603+0000 7fed7728f640 1 --2- 172.21.15.97:0/3608336767 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed7006b510 0x7fed7006b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.603+0000 7fed7728f640 1 -- 172.21.15.97:0/3608336767 >> 172.21.15.97:0/3608336767 conn(0x7fed700b6af0 msgr2=0x7fed7009aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.603+0000 7fed7728f640 1 -- 172.21.15.97:0/3608336767 shutdown_connections 2024-03-29T04:09:42.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.603+0000 7fed7728f640 1 -- 172.21.15.97:0/3608336767 wait complete. 2024-03-29T04:09:42.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.604+0000 7fed7728f640 1 Processor -- start 2024-03-29T04:09:42.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.604+0000 7fed7728f640 1 -- start start 2024-03-29T04:09:42.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7728f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7006b510 0x7fed70142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7728f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed700b2c10 0x7fed701610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7728f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed700d1100 0x7fed70161600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7728f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fed70145430 con 0x7fed7006b510 2024-03-29T04:09:42.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7728f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fed701452b0 con 0x7fed700d1100 2024-03-29T04:09:42.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7728f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fed700b48f0 con 0x7fed700b2c10 2024-03-29T04:09:42.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7502b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7006b510 0x7fed70142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7502b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7006b510 0x7fed70142cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39512/0 (socket says 172.21.15.97:39512) 2024-03-29T04:09:42.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7502b640 1 -- 172.21.15.97:0/2326024812 learned_addr learned my addr 172.21.15.97:0/2326024812 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:42.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7502b640 1 -- 172.21.15.97:0/2326024812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed700b2c10 msgr2=0x7fed701610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7502b640 1 --2- 172.21.15.97:0/2326024812 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fed700b2c10 0x7fed701610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7502b640 1 -- 172.21.15.97:0/2326024812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed700d1100 msgr2=0x7fed70161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7502b640 1 --2- 172.21.15.97:0/2326024812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed700d1100 0x7fed70161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7502b640 1 -- 172.21.15.97:0/2326024812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7009b6f0 con 0x7fed7006b510 2024-03-29T04:09:42.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.605+0000 7fed7502b640 1 --2- 172.21.15.97:0/2326024812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7006b510 0x7fed70142cb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00c820 tx=0x7fed6c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.606+0000 7fed65ffb640 1 -- 172.21.15.97:0/2326024812 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c015070 con 0x7fed7006b510 2024-03-29T04:09:42.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.606+0000 7fed65ffb640 1 -- 172.21.15.97:0/2326024812 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6c007a30 con 0x7fed7006b510 2024-03-29T04:09:42.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.606+0000 7fed65ffb640 1 -- 172.21.15.97:0/2326024812 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c007de0 con 0x7fed7006b510 2024-03-29T04:09:42.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.606+0000 7fed7728f640 1 -- 172.21.15.97:0/2326024812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7009b9d0 con 0x7fed7006b510 2024-03-29T04:09:42.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.606+0000 7fed7728f640 1 -- 172.21.15.97:0/2326024812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed700cac20 con 0x7fed7006b510 2024-03-29T04:09:42.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.607+0000 7fed7728f640 1 -- 172.21.15.97:0/2326024812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed7009b4a0 con 0x7fed7006b510 2024-03-29T04:09:42.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.607+0000 7fed65ffb640 1 -- 172.21.15.97:0/2326024812 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fed6c018070 con 0x7fed7006b510 2024-03-29T04:09:42.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.607+0000 7fed65ffb640 1 --2- 172.21.15.97:0/2326024812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fed5406c530 0x7fed5406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.608+0000 7fed65ffb640 1 -- 172.21.15.97:0/2326024812 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fed6c08b280 con 0x7fed7006b510 2024-03-29T04:09:42.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.608+0000 7fed7482a640 1 --2- 172.21.15.97:0/2326024812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fed5406c530 0x7fed5406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.608+0000 7fed65ffb640 1 --2- 172.21.15.97:0/2326024812 >> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] conn(0x7fed540723f0 0x7fed54074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.608+0000 7fed65ffb640 1 -- 172.21.15.97:0/2326024812 --> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed6c01de50 con 0x7fed540723f0 2024-03-29T04:09:42.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.608+0000 7fed65ffb640 1 -- 172.21.15.97:0/2326024812 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fed6c092050 con 0x7fed7006b510 2024-03-29T04:09:42.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.608+0000 7fed7482a640 1 --2- 172.21.15.97:0/2326024812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fed5406c530 0x7fed5406e9f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fed60036000 tx=0x7fed60030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.608+0000 7fed7582c640 1 --2- 172.21.15.97:0/2326024812 >> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] conn(0x7fed540723f0 0x7fed54074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.608+0000 7fed7582c640 1 --2- 172.21.15.97:0/2326024812 >> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] conn(0x7fed540723f0 0x7fed54074810 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.609+0000 7fed65ffb640 1 -- 172.21.15.97:0/2326024812 <== osd.6 v2:172.21.15.154:6800/1169845562 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed6c01de50 con 0x7fed540723f0 2024-03-29T04:09:42.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.622+0000 7fed7728f640 1 -- 172.21.15.97:0/2326024812 --> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fed700ca730 con 0x7fed540723f0 2024-03-29T04:09:42.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.623+0000 7fed65ffb640 1 -- 172.21.15.97:0/2326024812 <== osd.6 v2:172.21.15.154:6800/1169845562 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fed700ca730 con 0x7fed540723f0 2024-03-29T04:09:42.602 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:09:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:09:42.603 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:09:42.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.623+0000 7fed7728f640 1 -- 172.21.15.97:0/2326024812 >> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] conn(0x7fed540723f0 msgr2=0x7fed54074810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.623+0000 7fed7728f640 1 --2- 172.21.15.97:0/2326024812 >> [v2:172.21.15.154:6800/1169845562,v1:172.21.15.154:6801/1169845562] conn(0x7fed540723f0 0x7fed54074810 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.623+0000 7fed7728f640 1 -- 172.21.15.97:0/2326024812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fed5406c530 msgr2=0x7fed5406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.623+0000 7fed7728f640 1 --2- 172.21.15.97:0/2326024812 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fed5406c530 0x7fed5406e9f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fed60036000 tx=0x7fed60030000 comp rx=0 tx=0).stop 2024-03-29T04:09:42.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.623+0000 7fed7728f640 1 -- 172.21.15.97:0/2326024812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7006b510 msgr2=0x7fed70142cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.623+0000 7fed7728f640 1 --2- 172.21.15.97:0/2326024812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed7006b510 0x7fed70142cb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00c820 tx=0x7fed6c00cce0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.623+0000 7fed7582c640 1 -- 172.21.15.97:0/2326024812 reap_dead start 2024-03-29T04:09:42.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.623+0000 7fed7728f640 1 -- 172.21.15.97:0/2326024812 shutdown_connections 2024-03-29T04:09:42.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.623+0000 7fed7728f640 1 -- 172.21.15.97:0/2326024812 >> 172.21.15.97:0/2326024812 conn(0x7fed700b6af0 msgr2=0x7fed70143de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.623+0000 7fed7728f640 1 -- 172.21.15.97:0/2326024812 shutdown_connections 2024-03-29T04:09:42.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.623+0000 7fed7728f640 1 -- 172.21.15.97:0/2326024812 wait complete. 2024-03-29T04:09:42.613 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-29T04:09:42.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.734+0000 7fdbdb68f640 1 Processor -- start 2024-03-29T04:09:42.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.735+0000 7fdbdb68f640 1 -- start start 2024-03-29T04:09:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.735+0000 7fdbdb68f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdbd4156d60 0x7fdbd40d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.735+0000 7fdbdb68f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdbd40cfc30 0x7fdbd409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.735+0000 7fdbdb68f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd40d20f0 con 0x7fdbd415d710 2024-03-29T04:09:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.735+0000 7fdbdb68f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd415dc70 con 0x7fdbd40cfc30 2024-03-29T04:09:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.735+0000 7fdbdb68f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd40cee70 con 0x7fdbd4156d60 2024-03-29T04:09:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.736+0000 7fdbd8c2a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdbd40cfc30 0x7fdbd409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.736+0000 7fdbd8c2a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdbd40cfc30 0x7fdbd409aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39520/0 (socket says 172.21.15.97:39520) 2024-03-29T04:09:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.736+0000 7fdbd8c2a640 1 -- 172.21.15.97:0/435715887 learned_addr learned my addr 172.21.15.97:0/435715887 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.736+0000 7fdbd942b640 1 --2- 172.21.15.97:0/435715887 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdbd4156d60 0x7fdbd40d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.736+0000 7fdbd8c2a640 1 -- 172.21.15.97:0/435715887 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdbd4156d60 msgr2=0x7fdbd40d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.736+0000 7fdbd8c2a640 1 --2- 172.21.15.97:0/435715887 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdbd4156d60 0x7fdbd40d1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.736+0000 7fdbd8c2a640 1 -- 172.21.15.97:0/435715887 >> v1:172.21.15.154:6789/0 conn(0x7fdbd415d710 legacy=0x7fdbd41602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.736+0000 7fdbd8c2a640 1 -- 172.21.15.97:0/435715887 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd40cf150 con 0x7fdbd40cfc30 2024-03-29T04:09:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.737+0000 7fdbd8c2a640 1 --2- 172.21.15.97:0/435715887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd40cfc30 0x7fdbd409aa30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009870 tx=0x7fdbc402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a34449cf2776e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.737+0000 7fdbcbfff640 1 -- 172.21.15.97:0/435715887 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4034070 con 0x7fdbd40cfc30 2024-03-29T04:09:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.737+0000 7fdbcbfff640 1 -- 172.21.15.97:0/435715887 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc4030840 con 0x7fdbd40cfc30 2024-03-29T04:09:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.737+0000 7fdbcbfff640 1 -- 172.21.15.97:0/435715887 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4030c10 con 0x7fdbd40cfc30 2024-03-29T04:09:42.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.737+0000 7fdbdb68f640 1 -- 172.21.15.97:0/435715887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd40cfc30 msgr2=0x7fdbd409aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.737+0000 7fdbdb68f640 1 --2- 172.21.15.97:0/435715887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd40cfc30 0x7fdbd409aa30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009870 tx=0x7fdbc402b980 comp rx=0 tx=0).stop 2024-03-29T04:09:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.737+0000 7fdbdb68f640 1 -- 172.21.15.97:0/435715887 shutdown_connections 2024-03-29T04:09:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.737+0000 7fdbdb68f640 1 --2- 172.21.15.97:0/435715887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd40cfc30 0x7fdbd409aa30 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.737+0000 7fdbdb68f640 1 --2- 172.21.15.97:0/435715887 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdbd4156d60 0x7fdbd40d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.737+0000 7fdbdb68f640 1 -- 172.21.15.97:0/435715887 >> 172.21.15.97:0/435715887 conn(0x7fdbd40b3f30 msgr2=0x7fdbd40b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.738+0000 7fdbdb68f640 1 -- 172.21.15.97:0/435715887 shutdown_connections 2024-03-29T04:09:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.738+0000 7fdbdb68f640 1 -- 172.21.15.97:0/435715887 wait complete. 2024-03-29T04:09:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.739+0000 7fdbdb68f640 1 Processor -- start 2024-03-29T04:09:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.739+0000 7fdbdb68f640 1 -- start start 2024-03-29T04:09:42.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.739+0000 7fdbdb68f640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdbd40cfc30 0x7fdbd40b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.739+0000 7fdbdb68f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd4156d60 0x7fdbd409fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.739+0000 7fdbdb68f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd415d710 0x7fdbd4145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.739+0000 7fdbdb68f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4157140 con 0x7fdbd4156d60 2024-03-29T04:09:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.739+0000 7fdbdb68f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd40cf710 con 0x7fdbd415d710 2024-03-29T04:09:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.739+0000 7fdbdb68f640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd409b7f0 con 0x7fdbd40cfc30 2024-03-29T04:09:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.739+0000 7fdbd8c2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd4156d60 0x7fdbd409fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.739+0000 7fdbd8c2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd4156d60 0x7fdbd409fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39526/0 (socket says 172.21.15.97:39526) 2024-03-29T04:09:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.739+0000 7fdbd8c2a640 1 -- 172.21.15.97:0/2918989709 learned_addr learned my addr 172.21.15.97:0/2918989709 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbd942b640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdbd40cfc30 0x7fdbd40b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbd9c2c640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd415d710 0x7fdbd4145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbd8c2a640 1 -- 172.21.15.97:0/2918989709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdbd40cfc30 msgr2=0x7fdbd40b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbd8c2a640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdbd40cfc30 0x7fdbd40b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbd8c2a640 1 -- 172.21.15.97:0/2918989709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd415d710 msgr2=0x7fdbd4145da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbd8c2a640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd415d710 0x7fdbd4145da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbd8c2a640 1 -- 172.21.15.97:0/2918989709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd40b3000 con 0x7fdbd4156d60 2024-03-29T04:09:42.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbd9c2c640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd415d710 0x7fdbd4145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:42.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbd8c2a640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd4156d60 0x7fdbd409fde0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdbc402b9c0 tx=0x7fdbc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbc9ffb640 1 -- 172.21.15.97:0/2918989709 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4034040 con 0x7fdbd4156d60 2024-03-29T04:09:42.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbc9ffb640 1 -- 172.21.15.97:0/2918989709 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc4039d90 con 0x7fdbd4156d60 2024-03-29T04:09:42.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbc9ffb640 1 -- 172.21.15.97:0/2918989709 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4043630 con 0x7fdbd4156d60 2024-03-29T04:09:42.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbdb68f640 1 -- 172.21.15.97:0/2918989709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd40b32e0 con 0x7fdbd4156d60 2024-03-29T04:09:42.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.740+0000 7fdbdb68f640 1 -- 172.21.15.97:0/2918989709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd409c630 con 0x7fdbd4156d60 2024-03-29T04:09:42.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.741+0000 7fdbdb68f640 1 -- 172.21.15.97:0/2918989709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd4068640 con 0x7fdbd4156d60 2024-03-29T04:09:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.741+0000 7fdbc9ffb640 1 -- 172.21.15.97:0/2918989709 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdbc404a060 con 0x7fdbd4156d60 2024-03-29T04:09:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.742+0000 7fdbc9ffb640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdbb806c530 0x7fdbb806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.742+0000 7fdbc9ffb640 1 -- 172.21.15.97:0/2918989709 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fdbc40ad2c0 con 0x7fdbd4156d60 2024-03-29T04:09:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.742+0000 7fdbd942b640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdbb806c530 0x7fdbb806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.742+0000 7fdbd942b640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdbb806c530 0x7fdbb806e9f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009a30 tx=0x7fdbd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.746+0000 7fdbc9ffb640 1 -- 172.21.15.97:0/2918989709 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdbc403e070 con 0x7fdbd4156d60 2024-03-29T04:09:42.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.923+0000 7fdbdb68f640 1 -- 172.21.15.97:0/2918989709 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fdbd409bd30 con 0x7fdbb806c530 2024-03-29T04:09:42.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.923+0000 7fdbc9ffb640 1 -- 172.21.15.97:0/2918989709 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdbd409bd30 con 0x7fdbb806c530 2024-03-29T04:09:42.903 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 6 to deep-scrub 2024-03-29T04:09:42.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.925+0000 7fdbdb68f640 1 -- 172.21.15.97:0/2918989709 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdbb806c530 msgr2=0x7fdbb806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.925+0000 7fdbdb68f640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdbb806c530 0x7fdbb806e9f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009a30 tx=0x7fdbd0008040 comp rx=0 tx=0).stop 2024-03-29T04:09:42.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.925+0000 7fdbdb68f640 1 -- 172.21.15.97:0/2918989709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd4156d60 msgr2=0x7fdbd409fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.925+0000 7fdbdb68f640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd4156d60 0x7fdbd409fde0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdbc402b9c0 tx=0x7fdbc40046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.925+0000 7fdbdb68f640 1 -- 172.21.15.97:0/2918989709 shutdown_connections 2024-03-29T04:09:42.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.925+0000 7fdbdb68f640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fdbb806c530 0x7fdbb806e9f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.925+0000 7fdbdb68f640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd415d710 0x7fdbd4145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.925+0000 7fdbdb68f640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd4156d60 0x7fdbd409fde0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.925+0000 7fdbdb68f640 1 --2- 172.21.15.97:0/2918989709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdbd40cfc30 0x7fdbd40b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.925+0000 7fdbdb68f640 1 -- 172.21.15.97:0/2918989709 >> 172.21.15.97:0/2918989709 conn(0x7fdbd40b3f30 msgr2=0x7fdbd415ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.925+0000 7fdbdb68f640 1 -- 172.21.15.97:0/2918989709 shutdown_connections 2024-03-29T04:09:42.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:42.925+0000 7fdbdb68f640 1 -- 172.21.15.97:0/2918989709 wait complete. 2024-03-29T04:09:42.915 INFO:tasks.ceph:Scrubbing osd.7 2024-03-29T04:09:42.916 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:09:43.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.040+0000 7fa504b7c640 1 Processor -- start 2024-03-29T04:09:43.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.040+0000 7fa504b7c640 1 -- start start 2024-03-29T04:09:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.040+0000 7fa504b7c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa50015ab00 0x7fa5000b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.041+0000 7fa504b7c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa500072870 0x7fa50015ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.041+0000 7fa504b7c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5000ad660 con 0x7fa50015e7b0 2024-03-29T04:09:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.041+0000 7fa504b7c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7fa50015f150 con 0x7fa50015ab00 2024-03-29T04:09:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.041+0000 7fa504b7c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7fa50006d860 con 0x7fa500072870 2024-03-29T04:09:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.041+0000 7fa4fe59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa50015ab00 0x7fa5000b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.041+0000 7fa4fe59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa50015ab00 0x7fa5000b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39532/0 (socket says 172.21.15.97:39532) 2024-03-29T04:09:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.041+0000 7fa4fe59c640 1 -- 172.21.15.97:0/4080102874 learned_addr learned my addr 172.21.15.97:0/4080102874 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.041+0000 7fa4fed9d640 1 --2- 172.21.15.97:0/4080102874 >> v2:172.21.15.154:3300/0 conn(0x7fa500072870 0x7fa50015ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.041+0000 7fa4fe59c640 1 -- 172.21.15.97:0/4080102874 >> v2:172.21.15.154:3300/0 conn(0x7fa500072870 msgr2=0x7fa50015ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.041+0000 7fa4fe59c640 1 --2- 172.21.15.97:0/4080102874 >> v2:172.21.15.154:3300/0 conn(0x7fa500072870 0x7fa50015ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.041+0000 7fa4fe59c640 1 -- 172.21.15.97:0/4080102874 >> v1:172.21.15.154:6789/0 conn(0x7fa50015e7b0 legacy=0x7fa50006dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.041+0000 7fa4fe59c640 1 -- 172.21.15.97:0/4080102874 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5000b4ab0 con 0x7fa50015ab00 2024-03-29T04:09:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.042+0000 7fa4fe59c640 1 --2- 172.21.15.97:0/4080102874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa50015ab00 0x7fa5000b43f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa50015f8b0 tx=0x7fa4f002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3130d27236922a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.042+0000 7fa4fdd9b640 1 -- 172.21.15.97:0/4080102874 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f002e070 con 0x7fa50015ab00 2024-03-29T04:09:43.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.042+0000 7fa4fdd9b640 1 -- 172.21.15.97:0/4080102874 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4f0037070 con 0x7fa50015ab00 2024-03-29T04:09:43.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.042+0000 7fa4fdd9b640 1 -- 172.21.15.97:0/4080102874 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0031a00 con 0x7fa50015ab00 2024-03-29T04:09:43.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.042+0000 7fa504b7c640 1 -- 172.21.15.97:0/4080102874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa50015ab00 msgr2=0x7fa5000b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.042+0000 7fa504b7c640 1 --2- 172.21.15.97:0/4080102874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa50015ab00 0x7fa5000b43f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa50015f8b0 tx=0x7fa4f002b880 comp rx=0 tx=0).stop 2024-03-29T04:09:43.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.042+0000 7fa504b7c640 1 -- 172.21.15.97:0/4080102874 shutdown_connections 2024-03-29T04:09:43.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.042+0000 7fa504b7c640 1 --2- 172.21.15.97:0/4080102874 >> v2:172.21.15.154:3300/0 conn(0x7fa500072870 0x7fa50015ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.042+0000 7fa504b7c640 1 --2- 172.21.15.97:0/4080102874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa50015ab00 0x7fa5000b43f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.042+0000 7fa504b7c640 1 -- 172.21.15.97:0/4080102874 >> 172.21.15.97:0/4080102874 conn(0x7fa50006b6f0 msgr2=0x7fa50006bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:43.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.042+0000 7fa504b7c640 1 -- 172.21.15.97:0/4080102874 shutdown_connections 2024-03-29T04:09:43.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.042+0000 7fa504b7c640 1 -- 172.21.15.97:0/4080102874 wait complete. 2024-03-29T04:09:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.044+0000 7fa504b7c640 1 Processor -- start 2024-03-29T04:09:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.044+0000 7fa504b7c640 1 -- start start 2024-03-29T04:09:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.044+0000 7fa504b7c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa500072870 0x7fa500147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.044+0000 7fa504b7c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa50015ab00 0x7fa5000aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.044+0000 7fa504b7c640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa50015e7b0 0x7fa5001e5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.044+0000 7fa504b7c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5000b5070 con 0x7fa50015ab00 2024-03-29T04:09:43.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.044+0000 7fa504b7c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa50006db40 con 0x7fa500072870 2024-03-29T04:09:43.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.044+0000 7fa504b7c640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7fa50015aee0 con 0x7fa50015e7b0 2024-03-29T04:09:43.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.044+0000 7fa4f7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa50015ab00 0x7fa5000aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa4f7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa50015ab00 0x7fa5000aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39536/0 (socket says 172.21.15.97:39536) 2024-03-29T04:09:43.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa4f7fff640 1 -- 172.21.15.97:0/1305596843 learned_addr learned my addr 172.21.15.97:0/1305596843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:43.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa4fe59c640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa500072870 0x7fa500147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa4fed9d640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa50015e7b0 0x7fa5001e5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa4f7fff640 1 -- 172.21.15.97:0/1305596843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa50015e7b0 msgr2=0x7fa5001e5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa4f7fff640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa50015e7b0 0x7fa5001e5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa4f7fff640 1 -- 172.21.15.97:0/1305596843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa500072870 msgr2=0x7fa500147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa4f7fff640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa500072870 0x7fa500147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa4f7fff640 1 -- 172.21.15.97:0/1305596843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5001535b0 con 0x7fa50015ab00 2024-03-29T04:09:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa4f7fff640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa50015ab00 0x7fa5000aee40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800c430 tx=0x7fa4e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa4f6ffd640 1 -- 172.21.15.97:0/1305596843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8011790 con 0x7fa50015ab00 2024-03-29T04:09:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa4f6ffd640 1 -- 172.21.15.97:0/1305596843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e8017070 con 0x7fa50015ab00 2024-03-29T04:09:43.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa504b7c640 1 -- 172.21.15.97:0/1305596843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa500148530 con 0x7fa50015ab00 2024-03-29T04:09:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa4f6ffd640 1 -- 172.21.15.97:0/1305596843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8011bb0 con 0x7fa50015ab00 2024-03-29T04:09:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.045+0000 7fa504b7c640 1 -- 172.21.15.97:0/1305596843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5000adb20 con 0x7fa50015ab00 2024-03-29T04:09:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.046+0000 7fa504b7c640 1 -- 172.21.15.97:0/1305596843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5000ae4c0 con 0x7fa50015ab00 2024-03-29T04:09:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.046+0000 7fa4f6ffd640 1 -- 172.21.15.97:0/1305596843 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa4e801c460 con 0x7fa50015ab00 2024-03-29T04:09:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.047+0000 7fa4f6ffd640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa4e006c530 0x7fa4e006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.047+0000 7fa4f6ffd640 1 -- 172.21.15.97:0/1305596843 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fa4e808b250 con 0x7fa50015ab00 2024-03-29T04:09:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.047+0000 7fa4fe59c640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa4e006c530 0x7fa4e006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.047+0000 7fa4f6ffd640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] conn(0x7fa4e00723f0 0x7fa4e0074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.047+0000 7fa4f6ffd640 1 -- 172.21.15.97:0/1305596843 --> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4e8021e50 con 0x7fa4e00723f0 2024-03-29T04:09:43.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.047+0000 7fa4f6ffd640 1 -- 172.21.15.97:0/1305596843 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa4e8092050 con 0x7fa50015ab00 2024-03-29T04:09:43.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.047+0000 7fa4fed9d640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] conn(0x7fa4e00723f0 0x7fa4e0074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.047+0000 7fa4fe59c640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa4e006c530 0x7fa4e006e9f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0036000 tx=0x7fa4f003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.048+0000 7fa4fed9d640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] conn(0x7fa4e00723f0 0x7fa4e0074810 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.048+0000 7fa4f6ffd640 1 -- 172.21.15.97:0/1305596843 <== osd.7 v2:172.21.15.154:6816/2375296016 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4e8021e50 con 0x7fa4e00723f0 2024-03-29T04:09:43.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.061+0000 7fa504b7c640 1 -- 172.21.15.97:0/1305596843 --> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa50006e860 con 0x7fa4e00723f0 2024-03-29T04:09:43.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.062+0000 7fa4f6ffd640 1 -- 172.21.15.97:0/1305596843 <== osd.7 v2:172.21.15.154:6816/2375296016 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa50006e860 con 0x7fa4e00723f0 2024-03-29T04:09:43.041 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-03-29T04:09:43.041 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:09:43.042 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-03-29T04:09:43.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.062+0000 7fa504b7c640 1 -- 172.21.15.97:0/1305596843 >> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] conn(0x7fa4e00723f0 msgr2=0x7fa4e0074810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.062+0000 7fa504b7c640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.154:6816/2375296016,v1:172.21.15.154:6817/2375296016] conn(0x7fa4e00723f0 0x7fa4e0074810 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.062+0000 7fa504b7c640 1 -- 172.21.15.97:0/1305596843 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa4e006c530 msgr2=0x7fa4e006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.062+0000 7fa504b7c640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7fa4e006c530 0x7fa4e006e9f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0036000 tx=0x7fa4f003d040 comp rx=0 tx=0).stop 2024-03-29T04:09:43.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.062+0000 7fa504b7c640 1 -- 172.21.15.97:0/1305596843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa50015ab00 msgr2=0x7fa5000aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.062+0000 7fa504b7c640 1 --2- 172.21.15.97:0/1305596843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa50015ab00 0x7fa5000aee40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800c430 tx=0x7fa4e800c8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.062+0000 7fa4fed9d640 1 -- 172.21.15.97:0/1305596843 reap_dead start 2024-03-29T04:09:43.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.062+0000 7fa504b7c640 1 -- 172.21.15.97:0/1305596843 shutdown_connections 2024-03-29T04:09:43.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.062+0000 7fa504b7c640 1 -- 172.21.15.97:0/1305596843 >> 172.21.15.97:0/1305596843 conn(0x7fa50006b6f0 msgr2=0x7fa500156430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:43.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.062+0000 7fa504b7c640 1 -- 172.21.15.97:0/1305596843 shutdown_connections 2024-03-29T04:09:43.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.062+0000 7fa504b7c640 1 -- 172.21.15.97:0/1305596843 wait complete. 2024-03-29T04:09:43.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-29T04:09:43.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.174+0000 7f7867ec4640 1 Processor -- start 2024-03-29T04:09:43.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.175+0000 7f7867ec4640 1 -- start start 2024-03-29T04:09:43.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.175+0000 7f7867ec4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f786015ec10 0x7f78600c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.175+0000 7f7867ec4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78600d0730 0x7f78600b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.176+0000 7f7867ec4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f786015f100 con 0x7f78600ce740 2024-03-29T04:09:43.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.176+0000 7f7867ec4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f786015f370 con 0x7f786015ec10 2024-03-29T04:09:43.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.176+0000 7f7867ec4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f786015f650 con 0x7f78600d0730 2024-03-29T04:09:43.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.176+0000 7f7866461640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78600d0730 0x7f78600b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.176+0000 7f7865c60640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f786015ec10 0x7f78600c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.176+0000 7f7866461640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78600d0730 0x7f78600b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.97:51140/0 (socket says 172.21.15.97:51140) 2024-03-29T04:09:43.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.176+0000 7f7866461640 1 -- 172.21.15.97:0/2652486542 learned_addr learned my addr 172.21.15.97:0/2652486542 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:43.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.176+0000 7f7866461640 1 -- 172.21.15.97:0/2652486542 >> v2:172.21.15.154:3300/0 conn(0x7f786015ec10 msgr2=0x7f78600c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:43.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.176+0000 7f7866461640 1 --2- 172.21.15.97:0/2652486542 >> v2:172.21.15.154:3300/0 conn(0x7f786015ec10 0x7f78600c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.176+0000 7f7866461640 1 -- 172.21.15.97:0/2652486542 >> v1:172.21.15.154:6789/0 conn(0x7f78600ce740 legacy=0x7f78600d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.176+0000 7f7866461640 1 -- 172.21.15.97:0/2652486542 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78600cebd0 con 0x7f78600d0730 2024-03-29T04:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.177+0000 7f7866461640 1 --2- 172.21.15.97:0/2652486542 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78600d0730 0x7f78600b30b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7858009e50 tx=0x7f785802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3241c010ac4aad1e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.177+0000 7f7864c5e640 1 -- 172.21.15.97:0/2652486542 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785802ca70 con 0x7f78600d0730 2024-03-29T04:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.177+0000 7f7864c5e640 1 -- 172.21.15.97:0/2652486542 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f785802cc10 con 0x7f78600d0730 2024-03-29T04:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.177+0000 7f7864c5e640 1 -- 172.21.15.97:0/2652486542 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78580343f0 con 0x7f78600d0730 2024-03-29T04:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.178+0000 7f7867ec4640 1 -- 172.21.15.97:0/2652486542 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78600d0730 msgr2=0x7f78600b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.178+0000 7f7867ec4640 1 --2- 172.21.15.97:0/2652486542 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78600d0730 0x7f78600b30b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7858009e50 tx=0x7f785802bed0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.178+0000 7f7867ec4640 1 -- 172.21.15.97:0/2652486542 shutdown_connections 2024-03-29T04:09:43.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.178+0000 7f7867ec4640 1 --2- 172.21.15.97:0/2652486542 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78600d0730 0x7f78600b30b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.178+0000 7f7867ec4640 1 --2- 172.21.15.97:0/2652486542 >> v2:172.21.15.154:3300/0 conn(0x7f786015ec10 0x7f78600c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.178+0000 7f7867ec4640 1 -- 172.21.15.97:0/2652486542 >> 172.21.15.97:0/2652486542 conn(0x7f786006b510 msgr2=0x7f786006b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:43.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.178+0000 7f7867ec4640 1 -- 172.21.15.97:0/2652486542 shutdown_connections 2024-03-29T04:09:43.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.178+0000 7f7867ec4640 1 -- 172.21.15.97:0/2652486542 wait complete. 2024-03-29T04:09:43.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.179+0000 7f7867ec4640 1 Processor -- start 2024-03-29T04:09:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.179+0000 7f7867ec4640 1 -- start start 2024-03-29T04:09:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7867ec4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78600ce740 0x7f78600afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7867ec4640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78600d0730 0x7f78600727f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7867ec4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f786015ec10 0x7f786015c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7867ec4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f786015fe50 con 0x7f78600ce740 2024-03-29T04:09:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7867ec4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f78600c6ee0 con 0x7f786015ec10 2024-03-29T04:09:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7867ec4640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f786015ffd0 con 0x7f78600d0730 2024-03-29T04:09:43.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7865c60640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78600ce740 0x7f78600afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7865c60640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78600ce740 0x7f78600afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39544/0 (socket says 172.21.15.97:39544) 2024-03-29T04:09:43.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7865c60640 1 -- 172.21.15.97:0/2271381305 learned_addr learned my addr 172.21.15.97:0/2271381305 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:43.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f786545f640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78600d0730 0x7f78600727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7865c60640 1 -- 172.21.15.97:0/2271381305 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78600d0730 msgr2=0x7f78600727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7865c60640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78600d0730 0x7f78600727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7865c60640 1 -- 172.21.15.97:0/2271381305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f786015ec10 msgr2=0x7f786015c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7865c60640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f786015ec10 0x7f786015c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7865c60640 1 -- 172.21.15.97:0/2271381305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7860071750 con 0x7f78600ce740 2024-03-29T04:09:43.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f786545f640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78600d0730 0x7f78600727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:43.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7865c60640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78600ce740 0x7f78600afd20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f785000e9e0 tx=0x7f785000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f78567fc640 1 -- 172.21.15.97:0/2271381305 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850017070 con 0x7f78600ce740 2024-03-29T04:09:43.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f78567fc640 1 -- 172.21.15.97:0/2271381305 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f785000c9d0 con 0x7f78600ce740 2024-03-29T04:09:43.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7867ec4640 1 -- 172.21.15.97:0/2271381305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7860071a30 con 0x7f78600ce740 2024-03-29T04:09:43.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f78567fc640 1 -- 172.21.15.97:0/2271381305 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785000cdc0 con 0x7f78600ce740 2024-03-29T04:09:43.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.180+0000 7f7867ec4640 1 -- 172.21.15.97:0/2271381305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7860071ff0 con 0x7f78600ce740 2024-03-29T04:09:43.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.181+0000 7f7867ec4640 1 -- 172.21.15.97:0/2271381305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7860072430 con 0x7f78600ce740 2024-03-29T04:09:43.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.182+0000 7f78567fc640 1 -- 172.21.15.97:0/2271381305 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f785001a070 con 0x7f78600ce740 2024-03-29T04:09:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.182+0000 7f78567fc640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f784406c530 0x7f784406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.182+0000 7f78567fc640 1 -- 172.21.15.97:0/2271381305 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f785008b790 con 0x7f78600ce740 2024-03-29T04:09:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.182+0000 7f786545f640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f784406c530 0x7f784406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.183+0000 7f786545f640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f784406c530 0x7f784406e9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7860068f50 tx=0x7f785c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.186+0000 7f78567fc640 1 -- 172.21.15.97:0/2271381305 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f785005a0b0 con 0x7f78600ce740 2024-03-29T04:09:43.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.360+0000 7f7867ec4640 1 -- 172.21.15.97:0/2271381305 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f7860068640 con 0x7f784406c530 2024-03-29T04:09:43.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.360+0000 7f78567fc640 1 -- 172.21.15.97:0/2271381305 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7860068640 con 0x7f784406c530 2024-03-29T04:09:43.340 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 7 to deep-scrub 2024-03-29T04:09:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.362+0000 7f7867ec4640 1 -- 172.21.15.97:0/2271381305 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f784406c530 msgr2=0x7f784406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.362+0000 7f7867ec4640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f784406c530 0x7f784406e9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7860068f50 tx=0x7f785c009040 comp rx=0 tx=0).stop 2024-03-29T04:09:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.362+0000 7f7867ec4640 1 -- 172.21.15.97:0/2271381305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78600ce740 msgr2=0x7f78600afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.362+0000 7f7867ec4640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78600ce740 0x7f78600afd20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f785000e9e0 tx=0x7f785000eea0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.362+0000 7f7867ec4640 1 -- 172.21.15.97:0/2271381305 shutdown_connections 2024-03-29T04:09:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.362+0000 7f7867ec4640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f784406c530 0x7f784406e9f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.362+0000 7f7867ec4640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f786015ec10 0x7f786015c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.362+0000 7f7867ec4640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f78600d0730 0x7f78600727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.362+0000 7f7867ec4640 1 --2- 172.21.15.97:0/2271381305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78600ce740 0x7f78600afd20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.362+0000 7f7867ec4640 1 -- 172.21.15.97:0/2271381305 >> 172.21.15.97:0/2271381305 conn(0x7f786006b510 msgr2=0x7f786006b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:43.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.362+0000 7f7867ec4640 1 -- 172.21.15.97:0/2271381305 shutdown_connections 2024-03-29T04:09:43.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.362+0000 7f7867ec4640 1 -- 172.21.15.97:0/2271381305 wait complete. 2024-03-29T04:09:43.352 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:09:43.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.474+0000 7f9e882d3640 1 Processor -- start 2024-03-29T04:09:43.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.474+0000 7f9e882d3640 1 -- start start 2024-03-29T04:09:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.475+0000 7f9e882d3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9e8015b370 0x7f9e800b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.475+0000 7f9e882d3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e80071ae0 con 0x7f9e8006f3f0 2024-03-29T04:09:43.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.475+0000 7f9e882d3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e80071dc0 con 0x7f9e80073290 2024-03-29T04:09:43.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.475+0000 7f9e882d3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f9e800720a0 con 0x7f9e8015b370 2024-03-29T04:09:43.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.475+0000 7f9e8606f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9e8015b370 0x7f9e800b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.475+0000 7f9e8606f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9e8015b370 0x7f9e800b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39546/0 (socket says 172.21.15.97:39546) 2024-03-29T04:09:43.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.475+0000 7f9e8586e640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9e8006f3f0 0x7f9e80072b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41652/0 (socket says 172.21.15.97:41652) 2024-03-29T04:09:43.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.475+0000 7f9e8606f640 1 -- 172.21.15.97:0/3417482268 learned_addr learned my addr 172.21.15.97:0/3417482268 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:43.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.475+0000 7f9e8606f640 1 -- 172.21.15.97:0/3417482268 >> v1:172.21.15.154:6789/0 conn(0x7f9e80073290 legacy=0x7f9e8015f0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:09:43.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.475+0000 7f9e8606f640 1 -- 172.21.15.97:0/3417482268 >> v1:172.21.15.97:6789/0 conn(0x7f9e8006f3f0 legacy=0x7f9e80072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.475+0000 7f9e8606f640 1 -- 172.21.15.97:0/3417482268 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e80072380 con 0x7f9e8015b370 2024-03-29T04:09:43.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.475+0000 7f9e8506d640 1 -- 172.21.15.97:0/3417482268 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3300910018 0 0) 0x7f9e80071ae0 con 0x7f9e8006f3f0 2024-03-29T04:09:43.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.476+0000 7f9e8606f640 1 --2- 172.21.15.97:0/3417482268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8015b370 0x7f9e800b32c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00bb10 tx=0x7f9e7c02e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537b6720c7730ec5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.476+0000 7f9e8506d640 1 -- 172.21.15.97:0/3417482268 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c03b070 con 0x7f9e8015b370 2024-03-29T04:09:43.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.476+0000 7f9e8506d640 1 -- 172.21.15.97:0/3417482268 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7c036440 con 0x7f9e8015b370 2024-03-29T04:09:43.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.476+0000 7f9e8506d640 1 -- 172.21.15.97:0/3417482268 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c036810 con 0x7f9e8015b370 2024-03-29T04:09:43.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.477+0000 7f9e882d3640 1 -- 172.21.15.97:0/3417482268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8015b370 msgr2=0x7f9e800b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.477+0000 7f9e882d3640 1 --2- 172.21.15.97:0/3417482268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8015b370 0x7f9e800b32c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00bb10 tx=0x7f9e7c02e170 comp rx=0 tx=0).stop 2024-03-29T04:09:43.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.477+0000 7f9e882d3640 1 -- 172.21.15.97:0/3417482268 shutdown_connections 2024-03-29T04:09:43.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.477+0000 7f9e882d3640 1 --2- 172.21.15.97:0/3417482268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8015b370 0x7f9e800b32c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.477+0000 7f9e882d3640 1 -- 172.21.15.97:0/3417482268 >> 172.21.15.97:0/3417482268 conn(0x7f9e8006b510 msgr2=0x7f9e8006b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:43.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.477+0000 7f9e882d3640 1 -- 172.21.15.97:0/3417482268 shutdown_connections 2024-03-29T04:09:43.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.477+0000 7f9e882d3640 1 -- 172.21.15.97:0/3417482268 wait complete. 2024-03-29T04:09:43.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.478+0000 7f9e882d3640 1 Processor -- start 2024-03-29T04:09:43.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.478+0000 7f9e882d3640 1 -- start start 2024-03-29T04:09:43.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e882d3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8006f3f0 0x7f9e8014d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e882d3640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e80073290 0x7f9e8015c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e882d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e8015b370 0x7f9e8015dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e882d3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e800b3980 con 0x7f9e8006f3f0 2024-03-29T04:09:43.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e882d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9e800b3800 con 0x7f9e8015b370 2024-03-29T04:09:43.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e882d3640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9e8015b750 con 0x7f9e80073290 2024-03-29T04:09:43.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e8606f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8006f3f0 0x7f9e8014d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e8606f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8006f3f0 0x7f9e8014d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39562/0 (socket says 172.21.15.97:39562) 2024-03-29T04:09:43.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e8606f640 1 -- 172.21.15.97:0/2021076170 learned_addr learned my addr 172.21.15.97:0/2021076170 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:09:43.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e8586e640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e80073290 0x7f9e8015c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e86870640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e8015b370 0x7f9e8015dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e8606f640 1 -- 172.21.15.97:0/2021076170 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e80073290 msgr2=0x7f9e8015c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e8606f640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e80073290 0x7f9e8015c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e8606f640 1 -- 172.21.15.97:0/2021076170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e8015b370 msgr2=0x7f9e8015dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e8606f640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e8015b370 0x7f9e8015dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e8606f640 1 -- 172.21.15.97:0/2021076170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8015d380 con 0x7f9e8006f3f0 2024-03-29T04:09:43.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e86870640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e8015b370 0x7f9e8015dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:43.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e8606f640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8006f3f0 0x7f9e8014d4d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00b2c0 tx=0x7f9e7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e76ffd640 1 -- 172.21.15.97:0/2021076170 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c045070 con 0x7f9e8006f3f0 2024-03-29T04:09:43.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e76ffd640 1 -- 172.21.15.97:0/2021076170 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7c040890 con 0x7f9e8006f3f0 2024-03-29T04:09:43.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e882d3640 1 -- 172.21.15.97:0/2021076170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8014e540 con 0x7f9e8006f3f0 2024-03-29T04:09:43.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e76ffd640 1 -- 172.21.15.97:0/2021076170 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c03b040 con 0x7f9e8006f3f0 2024-03-29T04:09:43.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.479+0000 7f9e882d3640 1 -- 172.21.15.97:0/2021076170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8014eb00 con 0x7f9e8006f3f0 2024-03-29T04:09:43.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.480+0000 7f9e882d3640 1 -- 172.21.15.97:0/2021076170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e80068640 con 0x7f9e8006f3f0 2024-03-29T04:09:43.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.481+0000 7f9e76ffd640 1 -- 172.21.15.97:0/2021076170 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9e7c048070 con 0x7f9e8006f3f0 2024-03-29T04:09:43.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.481+0000 7f9e76ffd640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9e6406c530 0x7f9e6406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.481+0000 7f9e8586e640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9e6406c530 0x7f9e6406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.481+0000 7f9e76ffd640 1 -- 172.21.15.97:0/2021076170 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f9e7c0b0160 con 0x7f9e8006f3f0 2024-03-29T04:09:43.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.482+0000 7f9e8586e640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9e6406c530 0x7f9e6406e9f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9e8014da10 tx=0x7f9e70007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.486+0000 7f9e76ffd640 1 -- 172.21.15.97:0/2021076170 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9e7c07eb10 con 0x7f9e8006f3f0 2024-03-29T04:09:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.658+0000 7f9e882d3640 1 -- 172.21.15.97:0/2021076170 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e8006d9c0 con 0x7f9e6406c530 2024-03-29T04:09:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.659+0000 7f9e76ffd640 1 -- 172.21.15.97:0/2021076170 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13493 (secure 0 0 0) 0x7f9e8006d9c0 con 0x7f9e6406c530 2024-03-29T04:09:43.639 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:09:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-29T04:09:43.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.661+0000 7f9e882d3640 1 -- 172.21.15.97:0/2021076170 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9e6406c530 msgr2=0x7f9e6406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.661+0000 7f9e882d3640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9e6406c530 0x7f9e6406e9f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9e8014da10 tx=0x7f9e70007c00 comp rx=0 tx=0).stop 2024-03-29T04:09:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.661+0000 7f9e882d3640 1 -- 172.21.15.97:0/2021076170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8006f3f0 msgr2=0x7f9e8014d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.661+0000 7f9e882d3640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8006f3f0 0x7f9e8014d4d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00b2c0 tx=0x7f9e7c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.661+0000 7f9e882d3640 1 -- 172.21.15.97:0/2021076170 shutdown_connections 2024-03-29T04:09:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.661+0000 7f9e882d3640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9e6406c530 0x7f9e6406e9f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.661+0000 7f9e882d3640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e8015b370 0x7f9e8015dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.661+0000 7f9e882d3640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e80073290 0x7f9e8015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.661+0000 7f9e882d3640 1 --2- 172.21.15.97:0/2021076170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e8006f3f0 0x7f9e8014d4d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.661+0000 7f9e882d3640 1 -- 172.21.15.97:0/2021076170 >> 172.21.15.97:0/2021076170 conn(0x7f9e8006b510 msgr2=0x7f9e8006dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.661+0000 7f9e882d3640 1 -- 172.21.15.97:0/2021076170 shutdown_connections 2024-03-29T04:09:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:09:43.661+0000 7f9e882d3640 1 -- 172.21.15.97:0/2021076170 wait complete. 2024-03-29T04:09:43.651 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-03-29T04:09:42.659900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220512,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754208,"statfs":{"total":773094113280,"available":772868308992,"internally_reserved":0,"allocated":4431872,"data_stored":2446416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-173890432,"num_objects":-116,"num_object_clones":0,"num_object_copies":-232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-116,"num_whiteouts":0,"num_read":-75,"num_read_kb":0,"num_write":-193,"num_write_kb":-169857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.298468"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":199,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T04:09:35.370974+0000","last_change":"2024-03-29T04:07:29.856799+0000","last_active":"2024-03-29T04:09:35.370974+0000","last_peered":"2024-03-29T04:09:35.370974+0000","last_clean":"2024-03-29T04:09:35.370974+0000","last_became_active":"2024-03-29T04:07:29.855873+0000","last_became_peered":"2024-03-29T04:07:29.855873+0000","last_unstale":"2024-03-29T04:09:35.370974+0000","last_undegraded":"2024-03-29T04:09:35.370974+0000","last_fullsized":"2024-03-29T04:09:35.370974+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:07:28.661516+0000","last_clean_scrub_stamp":"2024-03-29T04:07:28.661516+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:25:15.944810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":748578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":158210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":158210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":748578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:09:43.653 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-29T04:07:28.661516+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=29, tm_hour=4, tm_min=7, tm_sec=28, tm_wday=4, tm_yday=89, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=29, tm_hour=4, tm_min=9, tm_sec=38, tm_wday=4, tm_yday=89, tm_isdst=0) 2024-03-29T04:09:43.653 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-29T04:10:03.654 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:10:03.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.790+0000 7f9e0f87e640 1 Processor -- start 2024-03-29T04:10:03.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.790+0000 7f9e0f87e640 1 -- start start 2024-03-29T04:10:03.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.791+0000 7f9e0f87e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e0815c670 0x7f9e0806c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.791+0000 7f9e0f87e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9e0806f060 0x7f9e0806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.791+0000 7f9e0f87e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e08146230 con 0x7f9e080ad230 2024-03-29T04:10:03.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.791+0000 7f9e0f87e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f9e0806dfc0 con 0x7f9e0806f060 2024-03-29T04:10:03.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.791+0000 7f9e0f87e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0806e2a0 con 0x7f9e0815c670 2024-03-29T04:10:03.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.791+0000 7f9e0de1b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9e080ad230 0x7f9e080b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46328/0 (socket says 172.21.15.97:46328) 2024-03-29T04:10:03.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.791+0000 7f9e0de1b640 1 -- 172.21.15.97:0/1089717559 learned_addr learned my addr 172.21.15.97:0/1089717559 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:10:03.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2010295407 0 0) 0x7f9e00001490 con 0x7f9e080ad230 2024-03-29T04:10:03.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9e0d61a640 1 --2- 172.21.15.97:0/1089717559 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e0815c670 0x7f9e0806c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9df4003520 con 0x7f9e080ad230 2024-03-29T04:10:03.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9e0ce19640 1 --2- 172.21.15.97:0/1089717559 >> v2:172.21.15.154:3300/0 conn(0x7f9e0806f060 0x7f9e0806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 501847349 0 0) 0x7f9df4003520 con 0x7f9e080ad230 2024-03-29T04:10:03.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e0815c670 msgr2=0x7f9e0806c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:03.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 --2- 172.21.15.97:0/1089717559 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e0815c670 0x7f9e0806c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 >> v2:172.21.15.154:3300/0 conn(0x7f9e0806f060 msgr2=0x7f9e0806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:03.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 --2- 172.21.15.97:0/1089717559 >> v2:172.21.15.154:3300/0 conn(0x7f9e0806f060 0x7f9e0806d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0806e580 con 0x7f9e080ad230 2024-03-29T04:10:03.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f9e00001bc0 con 0x7f9e080ad230 2024-03-29T04:10:03.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 >> v1:172.21.15.97:6789/0 conn(0x7f9e080ad230 legacy=0x7f9e080b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 --2- 172.21.15.97:0/1089717559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df40039b0 0x7f9df4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 --2- 172.21.15.97:0/1089717559 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e0815c670 0x7f9df4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 --2- 172.21.15.97:0/1089717559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df4004d40 0x7f9df40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0806dfc0 con 0x7f9df40039b0 2024-03-29T04:10:03.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0806e2a0 con 0x7f9df4004d40 2024-03-29T04:10:03.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9df400b9c0 con 0x7f9e0815c670 2024-03-29T04:10:03.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9e0d61a640 1 --2- 172.21.15.97:0/1089717559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df40039b0 0x7f9df4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9e0ce19640 1 --2- 172.21.15.97:0/1089717559 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e0815c670 0x7f9df4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9e0de1b640 1 --2- 172.21.15.97:0/1089717559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df4004d40 0x7f9df40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9e0d61a640 1 -- 172.21.15.97:0/1089717559 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e0815c670 msgr2=0x7f9df4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:03.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9e0d61a640 1 --2- 172.21.15.97:0/1089717559 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e0815c670 0x7f9df4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9e0d61a640 1 -- 172.21.15.97:0/1089717559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df4004d40 msgr2=0x7f9df40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:03.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9e0d61a640 1 --2- 172.21.15.97:0/1089717559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df4004d40 0x7f9df40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.792+0000 7f9e0d61a640 1 -- 172.21.15.97:0/1089717559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df4003520 con 0x7f9df40039b0 2024-03-29T04:10:03.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.793+0000 7f9e0d61a640 1 --2- 172.21.15.97:0/1089717559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df40039b0 0x7f9df4003dd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009ac0 tx=0x7f9df4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732a9b6717d85f26 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.793+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04039070 con 0x7f9df40039b0 2024-03-29T04:10:03.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.793+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e0402c6b0 con 0x7f9df40039b0 2024-03-29T04:10:03.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.793+0000 7f9dfffff640 1 -- 172.21.15.97:0/1089717559 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04009690 con 0x7f9df40039b0 2024-03-29T04:10:03.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.794+0000 7f9e0f87e640 1 -- 172.21.15.97:0/1089717559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df40039b0 msgr2=0x7f9df4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.794+0000 7f9e0f87e640 1 --2- 172.21.15.97:0/1089717559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df40039b0 0x7f9df4003dd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009ac0 tx=0x7f9df4005830 comp rx=0 tx=0).stop 2024-03-29T04:10:03.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.794+0000 7f9e0de1b640 1 -- 172.21.15.97:0/1089717559 reap_dead start 2024-03-29T04:10:03.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.794+0000 7f9e0f87e640 1 -- 172.21.15.97:0/1089717559 shutdown_connections 2024-03-29T04:10:03.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.794+0000 7f9e0f87e640 1 -- 172.21.15.97:0/1089717559 >> 172.21.15.97:0/1089717559 conn(0x7f9e0806b510 msgr2=0x7f9e0806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:03.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.794+0000 7f9e0f87e640 1 -- 172.21.15.97:0/1089717559 shutdown_connections 2024-03-29T04:10:03.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.794+0000 7f9e0f87e640 1 -- 172.21.15.97:0/1089717559 wait complete. 2024-03-29T04:10:03.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.795+0000 7f9e0f87e640 1 Processor -- start 2024-03-29T04:10:03.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.795+0000 7f9e0f87e640 1 -- start start 2024-03-29T04:10:03.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0f87e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df40039b0 0x7f9e08153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0f87e640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080b4600 0x7f9e08155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0f87e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e08155810 0x7f9e08156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0f87e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0806c030 con 0x7f9e08155810 2024-03-29T04:10:03.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0f87e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0815ca50 con 0x7f9df40039b0 2024-03-29T04:10:03.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0f87e640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0806c1b0 con 0x7f9e080b4600 2024-03-29T04:10:03.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0de1b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e08155810 0x7f9e08156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0de1b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e08155810 0x7f9e08156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52358/0 (socket says 172.21.15.97:52358) 2024-03-29T04:10:03.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0de1b640 1 -- 172.21.15.97:0/2153849154 learned_addr learned my addr 172.21.15.97:0/2153849154 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:10:03.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0ce19640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080b4600 0x7f9e08155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0d61a640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df40039b0 0x7f9e08153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0de1b640 1 -- 172.21.15.97:0/2153849154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080b4600 msgr2=0x7f9e08155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0de1b640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080b4600 0x7f9e08155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0de1b640 1 -- 172.21.15.97:0/2153849154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df40039b0 msgr2=0x7f9e08153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0de1b640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df40039b0 0x7f9e08153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.796+0000 7f9e0de1b640 1 -- 172.21.15.97:0/2153849154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e08154a90 con 0x7f9e08155810 2024-03-29T04:10:03.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.797+0000 7f9e0de1b640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e08155810 0x7f9e08156230 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9e0806f960 tx=0x7f9e0000af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.797+0000 7f9dfdffb640 1 -- 172.21.15.97:0/2153849154 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000077a0 con 0x7f9e08155810 2024-03-29T04:10:03.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.797+0000 7f9dfdffb640 1 -- 172.21.15.97:0/2153849154 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e00019070 con 0x7f9e08155810 2024-03-29T04:10:03.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.797+0000 7f9e0f87e640 1 -- 172.21.15.97:0/2153849154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0815dfa0 con 0x7f9e08155810 2024-03-29T04:10:03.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.797+0000 7f9dfdffb640 1 -- 172.21.15.97:0/2153849154 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000133f0 con 0x7f9e08155810 2024-03-29T04:10:03.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.797+0000 7f9e0f87e640 1 -- 172.21.15.97:0/2153849154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e081543f0 con 0x7f9e08155810 2024-03-29T04:10:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.798+0000 7f9e0f87e640 1 -- 172.21.15.97:0/2153849154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e08068640 con 0x7f9e08155810 2024-03-29T04:10:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.798+0000 7f9dfdffb640 1 -- 172.21.15.97:0/2153849154 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9e00013590 con 0x7f9e08155810 2024-03-29T04:10:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.799+0000 7f9dfdffb640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9dec06c530 0x7f9dec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.799+0000 7f9e0d61a640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9dec06c530 0x7f9dec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.799+0000 7f9dfdffb640 1 -- 172.21.15.97:0/2153849154 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f9e0008b980 con 0x7f9e08155810 2024-03-29T04:10:03.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.799+0000 7f9e0d61a640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9dec06c530 0x7f9dec06e9f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9e04036000 tx=0x7f9e04030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.803+0000 7f9dfdffb640 1 -- 172.21.15.97:0/2153849154 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9e08146230 con 0x7f9e08155810 2024-03-29T04:10:03.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.975+0000 7f9e0f87e640 1 -- 172.21.15.97:0/2153849154 --> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e08157c10 con 0x7f9dec06c530 2024-03-29T04:10:03.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.976+0000 7f9dfdffb640 1 -- 172.21.15.97:0/2153849154 <== mgr.4104 v2:172.21.15.97:6832/2286367679 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13355 (secure 0 0 0) 0x7f9e08157c10 con 0x7f9dec06c530 2024-03-29T04:10:03.956 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:10:03.956 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-03-29T04:10:03.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.978+0000 7f9e0f87e640 1 -- 172.21.15.97:0/2153849154 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9dec06c530 msgr2=0x7f9dec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.978+0000 7f9e0f87e640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9dec06c530 0x7f9dec06e9f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9e04036000 tx=0x7f9e04030000 comp rx=0 tx=0).stop 2024-03-29T04:10:03.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.978+0000 7f9e0f87e640 1 -- 172.21.15.97:0/2153849154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e08155810 msgr2=0x7f9e08156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.978+0000 7f9e0f87e640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e08155810 0x7f9e08156230 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9e0806f960 tx=0x7f9e0000af80 comp rx=0 tx=0).stop 2024-03-29T04:10:03.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.978+0000 7f9e0f87e640 1 -- 172.21.15.97:0/2153849154 shutdown_connections 2024-03-29T04:10:03.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.978+0000 7f9e0f87e640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f9dec06c530 0x7f9dec06e9f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.978+0000 7f9e0f87e640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e08155810 0x7f9e08156230 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.978+0000 7f9e0f87e640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e080b4600 0x7f9e08155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.978+0000 7f9e0f87e640 1 --2- 172.21.15.97:0/2153849154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df40039b0 0x7f9e08153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.978+0000 7f9e0f87e640 1 -- 172.21.15.97:0/2153849154 >> 172.21.15.97:0/2153849154 conn(0x7f9e0806b510 msgr2=0x7f9e081446b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.978+0000 7f9e0f87e640 1 -- 172.21.15.97:0/2153849154 shutdown_connections 2024-03-29T04:10:03.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:03.978+0000 7f9e0f87e640 1 -- 172.21.15.97:0/2153849154 wait complete. 2024-03-29T04:10:03.968 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-03-29T04:10:02.663582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220496,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754224,"statfs":{"total":773094113280,"available":772868325376,"internally_reserved":0,"allocated":4431872,"data_stored":2446416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002304"},"pg_stats":[{"pgid":"1.0","version":"35'119","reported_seq":210,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T04:09:43.553238+0000","last_change":"2024-03-29T04:09:43.553280+0000","last_active":"2024-03-29T04:09:43.553238+0000","last_peered":"2024-03-29T04:09:43.553238+0000","last_clean":"2024-03-29T04:09:43.553238+0000","last_became_active":"2024-03-29T04:07:29.855873+0000","last_became_peered":"2024-03-29T04:07:29.855873+0000","last_unstale":"2024-03-29T04:09:43.553238+0000","last_undegraded":"2024-03-29T04:09:43.553238+0000","last_fullsized":"2024-03-29T04:09:43.553238+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'119","last_scrub_stamp":"2024-03-29T04:09:43.553148+0000","last_deep_scrub":"35'119","last_deep_scrub_stamp":"2024-03-29T04:09:43.553148+0000","last_clean_scrub_stamp":"2024-03-29T04:09:43.553148+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:10:24.785336+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":999424,"data_stored":748578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":158210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":158210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":158210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":999424,"data_stored":748578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:10:03.969 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-29T04:10:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.091+0000 7f3f6d515640 1 Processor -- start 2024-03-29T04:10:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.091+0000 7f3f6d515640 1 -- start start 2024-03-29T04:10:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.091+0000 7f3f6d515640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3f680b4460 0x7f3f680ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.091+0000 7f3f6d515640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680b49b0 0x7f3f680b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.091+0000 7f3f6d515640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f680c70a0 con 0x7f3f6815ad60 2024-03-29T04:10:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.091+0000 7f3f6d515640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 -- 0x7f3f680c5ce0 con 0x7f3f680b4460 2024-03-29T04:10:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.091+0000 7f3f6d515640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3f680c5fc0 con 0x7f3f680b49b0 2024-03-29T04:10:04.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.092+0000 7f3f667fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3f680b4460 0x7f3f680ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.092+0000 7f3f667fc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3f680b4460 0x7f3f680ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52374/0 (socket says 172.21.15.97:52374) 2024-03-29T04:10:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.092+0000 7f3f667fc640 1 -- 172.21.15.97:0/4206324289 learned_addr learned my addr 172.21.15.97:0/4206324289 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:10:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.092+0000 7f3f677fe640 1 --2- 172.21.15.97:0/4206324289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680b49b0 0x7f3f680b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.092+0000 7f3f667fc640 1 -- 172.21.15.97:0/4206324289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680b49b0 msgr2=0x7f3f680b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.092+0000 7f3f667fc640 1 --2- 172.21.15.97:0/4206324289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680b49b0 0x7f3f680b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.092+0000 7f3f667fc640 1 -- 172.21.15.97:0/4206324289 >> v1:172.21.15.154:6789/0 conn(0x7f3f6815ad60 legacy=0x7f3f680c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.092+0000 7f3f667fc640 1 -- 172.21.15.97:0/4206324289 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f680cfd90 con 0x7f3f680b4460 2024-03-29T04:10:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.093+0000 7f3f667fc640 1 --2- 172.21.15.97:0/4206324289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f680b4460 0x7f3f680ce120 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009870 tx=0x7f3f5002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac2497a67c5b9ab server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.093+0000 7f3f65ffb640 1 -- 172.21.15.97:0/4206324289 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50035020 con 0x7f3f680b4460 2024-03-29T04:10:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.093+0000 7f3f65ffb640 1 -- 172.21.15.97:0/4206324289 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f50031640 con 0x7f3f680b4460 2024-03-29T04:10:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.093+0000 7f3f65ffb640 1 -- 172.21.15.97:0/4206324289 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5002d040 con 0x7f3f680b4460 2024-03-29T04:10:04.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.093+0000 7f3f6d515640 1 -- 172.21.15.97:0/4206324289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f680b4460 msgr2=0x7f3f680ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.093+0000 7f3f6d515640 1 --2- 172.21.15.97:0/4206324289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f680b4460 0x7f3f680ce120 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009870 tx=0x7f3f5002b8f0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.093+0000 7f3f6d515640 1 -- 172.21.15.97:0/4206324289 shutdown_connections 2024-03-29T04:10:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.093+0000 7f3f6d515640 1 --2- 172.21.15.97:0/4206324289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680b49b0 0x7f3f680b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.093+0000 7f3f6d515640 1 --2- 172.21.15.97:0/4206324289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f680b4460 0x7f3f680ce120 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.093+0000 7f3f6d515640 1 -- 172.21.15.97:0/4206324289 >> 172.21.15.97:0/4206324289 conn(0x7f3f680d0450 msgr2=0x7f3f680d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:04.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.093+0000 7f3f6d515640 1 -- 172.21.15.97:0/4206324289 shutdown_connections 2024-03-29T04:10:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.093+0000 7f3f6d515640 1 -- 172.21.15.97:0/4206324289 wait complete. 2024-03-29T04:10:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.095+0000 7f3f6d515640 1 Processor -- start 2024-03-29T04:10:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.095+0000 7f3f6d515640 1 -- start start 2024-03-29T04:10:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.095+0000 7f3f6d515640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f680b4460 0x7f3f681f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.095+0000 7f3f6d515640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680b49b0 0x7f3f681f3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.095+0000 7f3f6d515640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f6815ad60 0x7f3f68159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.095+0000 7f3f6d515640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3f680cfb30 con 0x7f3f6815ad60 2024-03-29T04:10:04.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.095+0000 7f3f6d515640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3f680cf9b0 con 0x7f3f680b4460 2024-03-29T04:10:04.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.095+0000 7f3f6d515640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f3f6806b510 con 0x7f3f680b49b0 2024-03-29T04:10:04.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.095+0000 7f3f677fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f6815ad60 0x7f3f68159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.095+0000 7f3f677fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f6815ad60 0x7f3f68159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52390/0 (socket says 172.21.15.97:52390) 2024-03-29T04:10:04.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.095+0000 7f3f677fe640 1 -- 172.21.15.97:0/2866848830 learned_addr learned my addr 172.21.15.97:0/2866848830 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:10:04.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f66ffd640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f680b4460 0x7f3f681f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f667fc640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680b49b0 0x7f3f681f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f677fe640 1 -- 172.21.15.97:0/2866848830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680b49b0 msgr2=0x7f3f681f3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f677fe640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680b49b0 0x7f3f681f3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f677fe640 1 -- 172.21.15.97:0/2866848830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f680b4460 msgr2=0x7f3f681f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f677fe640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f680b4460 0x7f3f681f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f677fe640 1 -- 172.21.15.97:0/2866848830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6815a290 con 0x7f3f6815ad60 2024-03-29T04:10:04.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f66ffd640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f680b4460 0x7f3f681f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:04.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f677fe640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f6815ad60 0x7f3f68159d50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800e850 tx=0x7f3f5800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f5f7fe640 1 -- 172.21.15.97:0/2866848830 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58017070 con 0x7f3f6815ad60 2024-03-29T04:10:04.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f5f7fe640 1 -- 172.21.15.97:0/2866848830 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f5800c940 con 0x7f3f6815ad60 2024-03-29T04:10:04.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f6d515640 1 -- 172.21.15.97:0/2866848830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f681f2990 con 0x7f3f6815ad60 2024-03-29T04:10:04.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f5f7fe640 1 -- 172.21.15.97:0/2866848830 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800cd30 con 0x7f3f6815ad60 2024-03-29T04:10:04.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.096+0000 7f3f6d515640 1 -- 172.21.15.97:0/2866848830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f681f2f50 con 0x7f3f6815ad60 2024-03-29T04:10:04.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.097+0000 7f3f6d515640 1 -- 172.21.15.97:0/2866848830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f681513c0 con 0x7f3f6815ad60 2024-03-29T04:10:04.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.097+0000 7f3f5f7fe640 1 -- 172.21.15.97:0/2866848830 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3f5801a070 con 0x7f3f6815ad60 2024-03-29T04:10:04.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.098+0000 7f3f5f7fe640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3f4806c530 0x7f3f4806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.098+0000 7f3f5f7fe640 1 -- 172.21.15.97:0/2866848830 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f3f5808b6b0 con 0x7f3f6815ad60 2024-03-29T04:10:04.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.098+0000 7f3f66ffd640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3f4806c530 0x7f3f4806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.098+0000 7f3f66ffd640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3f4806c530 0x7f3f4806e9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3f60001d60 tx=0x7f3f60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.102+0000 7f3f5f7fe640 1 -- 172.21.15.97:0/2866848830 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3f5805a060 con 0x7f3f6815ad60 2024-03-29T04:10:04.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.275+0000 7f3f6d515640 1 -- 172.21.15.97:0/2866848830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f3f6809e520 con 0x7f3f6815ad60 2024-03-29T04:10:04.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.278+0000 7f3f5f7fe640 1 -- 172.21.15.97:0/2866848830 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f3f58059960 con 0x7f3f6815ad60 2024-03-29T04:10:04.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.279+0000 7f3f5f7fe640 1 -- 172.21.15.97:0/2866848830 <== mon.0 v2:172.21.15.97:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3f5805dd10 con 0x7f3f6815ad60 2024-03-29T04:10:04.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.281+0000 7f3f6d515640 1 -- 172.21.15.97:0/2866848830 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3f4806c530 msgr2=0x7f3f4806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.281+0000 7f3f6d515640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3f4806c530 0x7f3f4806e9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3f60001d60 tx=0x7f3f60009040 comp rx=0 tx=0).stop 2024-03-29T04:10:04.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.281+0000 7f3f6d515640 1 -- 172.21.15.97:0/2866848830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f6815ad60 msgr2=0x7f3f68159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.281+0000 7f3f6d515640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f6815ad60 0x7f3f68159d50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800e850 tx=0x7f3f5800ed10 comp rx=0 tx=0).stop 2024-03-29T04:10:04.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.281+0000 7f3f6d515640 1 -- 172.21.15.97:0/2866848830 shutdown_connections 2024-03-29T04:10:04.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.281+0000 7f3f6d515640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f3f4806c530 0x7f3f4806e9f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.281+0000 7f3f6d515640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f6815ad60 0x7f3f68159d50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.281+0000 7f3f6d515640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3f680b49b0 0x7f3f681f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.281+0000 7f3f6d515640 1 --2- 172.21.15.97:0/2866848830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f680b4460 0x7f3f681f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.281+0000 7f3f6d515640 1 -- 172.21.15.97:0/2866848830 >> 172.21.15.97:0/2866848830 conn(0x7f3f680d0450 msgr2=0x7f3f680ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:04.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.281+0000 7f3f6d515640 1 -- 172.21.15.97:0/2866848830 shutdown_connections 2024-03-29T04:10:04.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.281+0000 7f3f6d515640 1 -- 172.21.15.97:0/2866848830 wait complete. 2024-03-29T04:10:04.272 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T04:10:04.272 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:10:04.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.393+0000 7f2109fe6640 1 Processor -- start 2024-03-29T04:10:04.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.393+0000 7f2109fe6640 1 -- start start 2024-03-29T04:10:04.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f2109fe6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f210406f3f0 0x7f2104072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f2109fe6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f21040730c0 0x7f210415f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f2109fe6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2104071ae0 con 0x7f210415b370 2024-03-29T04:10:04.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f2109fe6640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 -- 0x7f21040b39d0 con 0x7f21040730c0 2024-03-29T04:10:04.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f2109fe6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f2104071dc0 con 0x7f210406f3f0 2024-03-29T04:10:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f21037fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f210415b370 0x7f21040b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46336/0 (socket says 172.21.15.97:46336) 2024-03-29T04:10:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f21037fe640 1 -- 172.21.15.97:0/2225679937 learned_addr learned my addr 172.21.15.97:0/2225679937 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-03-29T04:10:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f2103fff640 1 --2- 172.21.15.97:0/2225679937 >> v2:172.21.15.154:3300/0 conn(0x7f21040730c0 0x7f210415f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f2102ffd640 1 --2- 172.21.15.97:0/2225679937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f210406f3f0 0x7f2104072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423143194 0 0) 0x7f2104071ae0 con 0x7f210415b370 2024-03-29T04:10:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20ec003520 con 0x7f210415b370 2024-03-29T04:10:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2471680206 0 0) 0x7f20ec003520 con 0x7f210415b370 2024-03-29T04:10:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f210406f3f0 msgr2=0x7f2104072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f21027fc640 1 --2- 172.21.15.97:0/2225679937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f210406f3f0 0x7f2104072b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 >> v2:172.21.15.154:3300/0 conn(0x7f21040730c0 msgr2=0x7f210415f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f21027fc640 1 --2- 172.21.15.97:0/2225679937 >> v2:172.21.15.154:3300/0 conn(0x7f21040730c0 0x7f210415f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21040720a0 con 0x7f210415b370 2024-03-29T04:10:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.394+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3366310308 0 0) 0x7f20f4001670 con 0x7f210415b370 2024-03-29T04:10:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 >> v1:172.21.15.97:6789/0 conn(0x7f210415b370 legacy=0x7f21040b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f21027fc640 1 --2- 172.21.15.97:0/2225679937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f210406f3f0 0x7f20ec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f21027fc640 1 --2- 172.21.15.97:0/2225679937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20ec003f30 0x7f20ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f21027fc640 1 --2- 172.21.15.97:0/2225679937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20ec004d40 0x7f20ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f21040b39d0 con 0x7f20ec003f30 2024-03-29T04:10:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2104071dc0 con 0x7f20ec004d40 2024-03-29T04:10:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec00dbd0 con 0x7f210406f3f0 2024-03-29T04:10:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f2102ffd640 1 --2- 172.21.15.97:0/2225679937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20ec003f30 0x7f20ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f21037fe640 1 --2- 172.21.15.97:0/2225679937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f210406f3f0 0x7f20ec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f2103fff640 1 --2- 172.21.15.97:0/2225679937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20ec004d40 0x7f20ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f2102ffd640 1 -- 172.21.15.97:0/2225679937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f210406f3f0 msgr2=0x7f20ec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f2102ffd640 1 --2- 172.21.15.97:0/2225679937 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f210406f3f0 0x7f20ec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f2102ffd640 1 -- 172.21.15.97:0/2225679937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20ec004d40 msgr2=0x7f20ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f2102ffd640 1 --2- 172.21.15.97:0/2225679937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20ec004d40 0x7f20ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.395+0000 7f2102ffd640 1 -- 172.21.15.97:0/2225679937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec003520 con 0x7f20ec003f30 2024-03-29T04:10:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.396+0000 7f2102ffd640 1 --2- 172.21.15.97:0/2225679937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20ec003f30 0x7f20ec004800 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009ac0 tx=0x7f20ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749826138f78bbfa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.396+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8039070 con 0x7f20ec003f30 2024-03-29T04:10:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.396+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f20f802c6b0 con 0x7f20ec003f30 2024-03-29T04:10:04.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.396+0000 7f21027fc640 1 -- 172.21.15.97:0/2225679937 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8009690 con 0x7f20ec003f30 2024-03-29T04:10:04.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.396+0000 7f2109fe6640 1 -- 172.21.15.97:0/2225679937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20ec003f30 msgr2=0x7f20ec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.396+0000 7f2109fe6640 1 --2- 172.21.15.97:0/2225679937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20ec003f30 0x7f20ec004800 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009ac0 tx=0x7f20ec005830 comp rx=0 tx=0).stop 2024-03-29T04:10:04.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.396+0000 7f2103fff640 1 -- 172.21.15.97:0/2225679937 reap_dead start 2024-03-29T04:10:04.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.396+0000 7f2109fe6640 1 -- 172.21.15.97:0/2225679937 shutdown_connections 2024-03-29T04:10:04.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.396+0000 7f2109fe6640 1 -- 172.21.15.97:0/2225679937 >> 172.21.15.97:0/2225679937 conn(0x7f210406b510 msgr2=0x7f210406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.396+0000 7f2109fe6640 1 -- 172.21.15.97:0/2225679937 shutdown_connections 2024-03-29T04:10:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.396+0000 7f2109fe6640 1 -- 172.21.15.97:0/2225679937 wait complete. 2024-03-29T04:10:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.397+0000 7f2109fe6640 1 Processor -- start 2024-03-29T04:10:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.398+0000 7f2109fe6640 1 -- start start 2024-03-29T04:10:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.398+0000 7f2109fe6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20ec003f30 0x7f21041f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.398+0000 7f2109fe6640 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21040cf6f0 0x7f21040ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.398+0000 7f2109fe6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21041575a0 0x7f2104157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.398+0000 7f2109fe6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2104073550 con 0x7f20ec003f30 2024-03-29T04:10:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.398+0000 7f2109fe6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2104072660 con 0x7f21041575a0 2024-03-29T04:10:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.398+0000 7f2109fe6640 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 -- 0x7f210415b750 con 0x7f21040cf6f0 2024-03-29T04:10:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.398+0000 7f21037fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20ec003f30 0x7f21041f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.398+0000 7f21037fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20ec003f30 0x7f21041f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52408/0 (socket says 172.21.15.97:52408) 2024-03-29T04:10:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.398+0000 7f21037fe640 1 -- 172.21.15.97:0/1359551051 learned_addr learned my addr 172.21.15.97:0/1359551051 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-03-29T04:10:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.399+0000 7f2103fff640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21041575a0 0x7f2104157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.399+0000 7f21037fe640 1 -- 172.21.15.97:0/1359551051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21040cf6f0 msgr2=0x7f21040ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.399+0000 7f21037fe640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21040cf6f0 0x7f21040ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.399+0000 7f21037fe640 1 -- 172.21.15.97:0/1359551051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21041575a0 msgr2=0x7f2104157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.399+0000 7f21037fe640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21041575a0 0x7f2104157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.399+0000 7f21037fe640 1 -- 172.21.15.97:0/1359551051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21040cfb70 con 0x7f20ec003f30 2024-03-29T04:10:04.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.399+0000 7f2103fff640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21041575a0 0x7f2104157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:04.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.399+0000 7f21037fe640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20ec003f30 0x7f21041f15c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f20f4001200 tx=0x7f20f400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.399+0000 7f20ebfff640 1 -- 172.21.15.97:0/1359551051 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f400ee00 con 0x7f20ec003f30 2024-03-29T04:10:04.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.399+0000 7f20ebfff640 1 -- 172.21.15.97:0/1359551051 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f20f4018070 con 0x7f20ec003f30 2024-03-29T04:10:04.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.399+0000 7f2109fe6640 1 -- 172.21.15.97:0/1359551051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21041594b0 con 0x7f20ec003f30 2024-03-29T04:10:04.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.399+0000 7f20ebfff640 1 -- 172.21.15.97:0/1359551051 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f400cad0 con 0x7f20ec003f30 2024-03-29T04:10:04.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.399+0000 7f2109fe6640 1 -- 172.21.15.97:0/1359551051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21041f4420 con 0x7f20ec003f30 2024-03-29T04:10:04.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.400+0000 7f2109fe6640 1 -- 172.21.15.97:0/1359551051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2104159680 con 0x7f20ec003f30 2024-03-29T04:10:04.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.400+0000 7f20ebfff640 1 -- 172.21.15.97:0/1359551051 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f20f400cc70 con 0x7f20ec003f30 2024-03-29T04:10:04.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.401+0000 7f20ebfff640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f20f006c560 0x7f20f006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.401+0000 7f20ebfff640 1 -- 172.21.15.97:0/1359551051 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f20f408b270 con 0x7f20ec003f30 2024-03-29T04:10:04.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.401+0000 7f2102ffd640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f20f006c560 0x7f20f006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.401+0000 7f2102ffd640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f20f006c560 0x7f20f006ea20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f20f8036000 tx=0x7f20f802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.405+0000 7f20ebfff640 1 -- 172.21.15.97:0/1359551051 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f20f4059b00 con 0x7f20ec003f30 2024-03-29T04:10:04.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.617+0000 7f2109fe6640 1 -- 172.21.15.97:0/1359551051 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f210406eaa0 con 0x7f20ec003f30 2024-03-29T04:10:04.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.618+0000 7f20ebfff640 1 -- 172.21.15.97:0/1359551051 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3872 (secure 0 0 0) 0x7f210406eaa0 con 0x7f20ec003f30 2024-03-29T04:10:04.597 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-03-29T04:10:04.598 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2199898540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":2199898540},{"type":"v1","addr":"172.21.15.97:6837","nonce":2199898540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1223352413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1223352413},{"type":"v1","addr":"172.21.15.154:6834","nonce":1223352413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/830866132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":830866132},{"type":"v1","addr":"172.21.15.154:6835","nonce":830866132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2079574241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2079574241},{"type":"v1","addr":"172.21.15.97:6835","nonce":2079574241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-03-29T04:10:04.598 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2024-03-29T04:10:04.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.620+0000 7f2109fe6640 1 -- 172.21.15.97:0/1359551051 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f20f006c560 msgr2=0x7f20f006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.620+0000 7f2109fe6640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f20f006c560 0x7f20f006ea20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f20f8036000 tx=0x7f20f802e000 comp rx=0 tx=0).stop 2024-03-29T04:10:04.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.620+0000 7f2109fe6640 1 -- 172.21.15.97:0/1359551051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20ec003f30 msgr2=0x7f21041f15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.620+0000 7f2109fe6640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20ec003f30 0x7f21041f15c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f20f4001200 tx=0x7f20f400ec10 comp rx=0 tx=0).stop 2024-03-29T04:10:04.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.620+0000 7f2109fe6640 1 -- 172.21.15.97:0/1359551051 shutdown_connections 2024-03-29T04:10:04.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.620+0000 7f2109fe6640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.97:6832/2286367679,v1:172.21.15.97:6833/2286367679] conn(0x7f20f006c560 0x7f20f006ea20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.620+0000 7f2109fe6640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f21041575a0 0x7f2104157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.620+0000 7f2109fe6640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f21040cf6f0 0x7f21040ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.620+0000 7f2109fe6640 1 --2- 172.21.15.97:0/1359551051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20ec003f30 0x7f21041f15c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.620+0000 7f2109fe6640 1 -- 172.21.15.97:0/1359551051 >> 172.21.15.97:0/1359551051 conn(0x7f210406b510 msgr2=0x7f21040b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:04.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.620+0000 7f2109fe6640 1 -- 172.21.15.97:0/1359551051 shutdown_connections 2024-03-29T04:10:04.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-03-29T04:10:04.620+0000 7f2109fe6640 1 -- 172.21.15.97:0/1359551051 wait complete. 2024-03-29T04:10:04.610 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-29T04:10:04.610 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-29T04:10:04.611 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T04:10:04.611 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:04.621 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T04:10:04.621 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T04:10:04.621 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:04.671 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T04:10:04.671 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T04:10:04.671 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:04.683 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T04:10:04.684 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T04:10:04.684 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:04.737 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T04:10:04.737 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-29T04:10:04.737 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-29T04:10:04.737 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:04.769 INFO:tasks.ceph.osd.0:Stopped 2024-03-29T04:10:04.769 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-29T04:10:04.769 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:04.834 INFO:tasks.ceph.osd.1:Stopped 2024-03-29T04:10:04.834 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-29T04:10:04.834 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:04.898 INFO:tasks.ceph.osd.2:Stopped 2024-03-29T04:10:04.898 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-29T04:10:04.899 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:04.971 INFO:tasks.ceph.osd.3:Stopped 2024-03-29T04:10:04.971 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-29T04:10:04.971 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:05.009 INFO:tasks.ceph.osd.4:Stopped 2024-03-29T04:10:05.009 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-29T04:10:05.009 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:05.088 INFO:tasks.ceph.osd.5:Stopped 2024-03-29T04:10:05.088 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-29T04:10:05.088 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:05.158 INFO:tasks.ceph.osd.6:Stopped 2024-03-29T04:10:05.159 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-29T04:10:05.159 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:05.231 INFO:tasks.ceph.osd.7:Stopped 2024-03-29T04:10:05.231 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-29T04:10:05.231 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-29T04:10:05.231 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:05.257 INFO:tasks.ceph.mgr.y:Stopped 2024-03-29T04:10:05.257 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-29T04:10:05.257 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:05.300 INFO:tasks.ceph.mgr.x:Stopped 2024-03-29T04:10:05.300 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-29T04:10:05.300 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-29T04:10:05.300 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:05.311 INFO:tasks.ceph.mon.a:Stopped 2024-03-29T04:10:05.311 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-29T04:10:05.311 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:05.353 INFO:tasks.ceph.mon.b:Stopped 2024-03-29T04:10:05.353 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-29T04:10:05.354 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:05.406 INFO:tasks.ceph.mon.c:Stopped 2024-03-29T04:10:05.406 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-29T04:10:05.407 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-03-29T04:10:05.447 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T04:10:05.447 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-29T04:10:05.664 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T04:10:05.664 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-29T04:10:05.747 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T04:10:05.748 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-29T04:10:05.797 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T04:10:05.797 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-29T04:10:05.889 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T04:10:05.889 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-29T04:10:06.079 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T04:10:06.080 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-29T04:10:06.141 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T04:10:06.141 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-29T04:10:06.199 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T04:10:06.200 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-29T04:10:06.258 INFO:tasks.ceph:Archiving mon data... 2024-03-29T04:10:06.259 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076/data/mon.a.tgz 2024-03-29T04:10:06.259 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-03-29T04:10:06.275 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.qCaFTWbMhO 2024-03-29T04:10:06.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.qCaFTWbMhO 2024-03-29T04:10:06.394 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.qCaFTWbMhO 2024-03-29T04:10:06.478 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.qCaFTWbMhO is 288KB 2024-03-29T04:10:06.573 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.qCaFTWbMhO 2024-03-29T04:10:06.590 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076/data/mon.b.tgz 2024-03-29T04:10:06.590 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-03-29T04:10:06.607 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.aeHJqSv2U0 2024-03-29T04:10:06.608 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.aeHJqSv2U0 2024-03-29T04:10:06.756 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.aeHJqSv2U0 2024-03-29T04:10:06.843 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.aeHJqSv2U0 is 384KB 2024-03-29T04:10:06.941 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.aeHJqSv2U0 2024-03-29T04:10:06.955 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076/data/mon.c.tgz 2024-03-29T04:10:06.956 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-03-29T04:10:07.009 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.BZexm2Kquf 2024-03-29T04:10:07.010 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.BZexm2Kquf 2024-03-29T04:10:07.146 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.BZexm2Kquf 2024-03-29T04:10:07.231 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.BZexm2Kquf is 383KB 2024-03-29T04:10:07.299 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.BZexm2Kquf 2024-03-29T04:10:07.314 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-29T04:10:07.314 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-29T04:10:07.317 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-29T04:10:07.400 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-29T04:10:07.436 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-29T04:10:07.436 INFO:tasks.ceph:Archiving crash dumps... 2024-03-29T04:10:07.438 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076/remote/smithi097/crash 2024-03-29T04:10:07.438 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-29T04:10:07.467 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076/remote/smithi154/crash 2024-03-29T04:10:07.468 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-29T04:10:07.502 INFO:tasks.ceph:Shutting down logrotate 2024-03-29T04:10:07.502 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:10:07.512 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:10:07.602 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-29T04:10:07.630 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-29T04:10:07.657 INFO:tasks.ceph:Compressing logs... 2024-03-29T04:10:07.657 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-29T04:10:07.672 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-29T04:10:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52762.log 2024-03-29T04:10:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44605.log 2024-03-29T04:10:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47578.log 2024-03-29T04:10:07.749 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37867.log 2024-03-29T04:10:07.750 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-29T04:10:07.750 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.37867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37827.log 2024-03-29T04:10:07.750 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.6.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37867.log.gz -5 2024-03-29T04:10:07.750 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.1.44422.log 2024-03-29T04:10:07.750 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.37827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37787.log 2024-03-29T04:10:07.750 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.1.44422.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37827.log.gz 2024-03-29T04:10:07.750 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.37947.log 2024-03-29T04:10:07.751 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.1.44422.log.gz 2024-03-29T04:10:07.751 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.37787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44819.log 2024-03-29T04:10:07.751 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.37947.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-29T04:10:07.751 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37787.log.gz 2024-03-29T04:10:07.751 INFO:teuthology.orchestra.run.smithi154.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37947.log.gz 2024-03-29T04:10:07.751 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.3.44819.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-29T04:10:07.751 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.44819.log.gz 2024-03-29T04:10:07.752 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37707.log 2024-03-29T04:10:07.752 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37907.log 2024-03-29T04:10:07.752 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.37707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45415.log 2024-03-29T04:10:07.752 INFO:teuthology.orchestra.run.smithi154.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37707.log.gz 2024-03-29T04:10:07.752 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.37907.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-29T04:10:07.753 INFO:teuthology.orchestra.run.smithi154.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-29T04:10:07.753 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37747.log 2024-03-29T04:10:07.753 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.1.45415.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37907.log.gz 2024-03-29T04:10:07.753 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45415.log.gz 2024-03-29T04:10:07.753 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-29T04:10:07.753 INFO:teuthology.orchestra.run.smithi154.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-29T04:10:07.753 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.37747.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-29T04:10:07.753 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37747.log.gz 2024-03-29T04:10:07.754 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37667.log 2024-03-29T04:10:07.754 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-29T04:10:07.754 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.37667.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-29T04:10:07.754 INFO:teuthology.orchestra.run.smithi154.stderr: 87.7%/var/log/ceph/ceph-mgr.x.log: -- replaced with /var/log/ceph/ceph-client.admin.37667.log.gz 2024-03-29T04:10:07.754 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-29T04:10:07.754 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-29T04:10:07.754 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-29T04:10:07.754 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mds.d.log: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-29T04:10:07.754 INFO:teuthology.orchestra.run.smithi154.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-29T04:10:07.784 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mon.c.log: 94.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52762.log.gz 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49099.log 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44605.log: /var/log/ceph/ceph-client.admin.47578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47410.log 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44605.log.gz 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49099.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47578.log.gz 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49765.log 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49099.log.gz 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43441.log 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4%/var/log/ceph/ceph-client.admin.49765.log: -- replaced with /var/log/ceph/ceph-client.admin.47410.log.gz 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51894.log 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49638.log 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49765.log.gz 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51894.log: gzip -5 --verbose -- 88.9% /var/log/ceph/ceph-client.admin.49435.log 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43441.log.gz 2024-03-29T04:10:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49638.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-29T04:10:07.796 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51894.log.gz 2024-03-29T04:10:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49435.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49638.log.gz 2024-03-29T04:10:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50244.log 2024-03-29T04:10:07.796 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49435.log.gz 2024-03-29T04:10:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43631.log 2024-03-29T04:10:07.796 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43399.log 2024-03-29T04:10:07.797 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44274.log 2024-03-29T04:10:07.797 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50244.log.gz 2024-03-29T04:10:07.797 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43673.log 2024-03-29T04:10:07.797 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43631.log.gz 2024-03-29T04:10:07.797 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43191.log 2024-03-29T04:10:07.797 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43399.log.gz 2024-03-29T04:10:07.797 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44274.log.gz 2024-03-29T04:10:07.797 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43925.log 2024-03-29T04:10:07.798 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43673.log.gz 2024-03-29T04:10:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48001.log 2024-03-29T04:10:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43925.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43191.log.gz 2024-03-29T04:10:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44093.log 2024-03-29T04:10:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48001.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43925.log.gz 2024-03-29T04:10:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47899.log 2024-03-29T04:10:07.798 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48001.log.gz 2024-03-29T04:10:07.799 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49849.log 2024-03-29T04:10:07.799 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47899.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44093.log.gz 2024-03-29T04:10:07.799 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46906.log 2024-03-29T04:10:07.799 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47899.log.gz 2024-03-29T04:10:07.799 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52805.log 2024-03-29T04:10:07.799 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49849.log.gz 2024-03-29T04:10:07.799 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45605.log 2024-03-29T04:10:07.799 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46906.log.gz 2024-03-29T04:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48163.log 2024-03-29T04:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49891.log 2024-03-29T04:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52805.log.gz 2024-03-29T04:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48163.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45605.log.gz 2024-03-29T04:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46384.log 2024-03-29T04:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48163.log.gz 2024-03-29T04:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52408.log 2024-03-29T04:10:07.801 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49891.log.gz 2024-03-29T04:10:07.801 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44941.log 2024-03-29T04:10:07.801 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52408.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46384.log.gz 2024-03-29T04:10:07.801 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45731.log 2024-03-29T04:10:07.801 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52408.log.gz 2024-03-29T04:10:07.801 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50496.log 2024-03-29T04:10:07.801 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44941.log.gz 2024-03-29T04:10:07.801 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51978.log 2024-03-29T04:10:07.802 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50496.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45731.log.gz 2024-03-29T04:10:07.802 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43095.log 2024-03-29T04:10:07.803 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51978.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50496.log.gz 2024-03-29T04:10:07.803 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49393.log 2024-03-29T04:10:07.803 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8%/var/log/ceph/ceph-client.admin.43095.log: -- replaced with /var/log/ceph/ceph-client.admin.51978.log.gz 2024-03-29T04:10:07.803 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52450.log 2024-03-29T04:10:07.803 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49393.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43095.log.gz 2024-03-29T04:10:07.803 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53823.log 2024-03-29T04:10:07.803 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49393.log.gz 2024-03-29T04:10:07.804 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49596.log 2024-03-29T04:10:07.804 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52450.log.gz/var/log/ceph/ceph-client.admin.53823.log: 2024-03-29T04:10:07.804 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53205.log 2024-03-29T04:10:07.804 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50061.log 2024-03-29T04:10:07.804 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53823.log.gz 2024-03-29T04:10:07.804 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53205.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49596.log.gz 2024-03-29T04:10:07.804 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47536.log 2024-03-29T04:10:07.804 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50061.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53205.log.gz 2024-03-29T04:10:07.804 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48548.log 2024-03-29T04:10:07.805 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47536.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50061.log.gz 2024-03-29T04:10:07.805 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37563.log 2024-03-29T04:10:07.805 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3%/var/log/ceph/ceph-client.admin.48548.log: -- replaced with /var/log/ceph/ceph-client.admin.47536.log.gz 2024-03-29T04:10:07.805 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43272.log 2024-03-29T04:10:07.805 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48548.log.gz 2024-03-29T04:10:07.805 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43003.log/var/log/ceph/ceph-client.admin.37563.log: 2024-03-29T04:10:07.805 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44265.log 2024-03-29T04:10:07.805 INFO:teuthology.orchestra.run.smithi097.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37563.log.gz 2024-03-29T04:10:07.806 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43003.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43272.log.gz 2024-03-29T04:10:07.806 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48721.log 2024-03-29T04:10:07.806 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44265.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43003.log.gz 2024-03-29T04:10:07.806 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43314.log 2024-03-29T04:10:07.806 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48721.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44265.log.gz 2024-03-29T04:10:07.806 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53595.log 2024-03-29T04:10:07.806 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48721.log.gz 2024-03-29T04:10:07.806 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53781.log 2024-03-29T04:10:07.807 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48506.log 2024-03-29T04:10:07.807 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43314.log.gz 2024-03-29T04:10:07.807 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53781.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53595.log.gz 2024-03-29T04:10:07.807 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54116.log 2024-03-29T04:10:07.807 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53781.log.gz 2024-03-29T04:10:07.807 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52366.log 2024-03-29T04:10:07.807 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48463.log 2024-03-29T04:10:07.807 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48506.log.gz 2024-03-29T04:10:07.807 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52366.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54116.log.gz 2024-03-29T04:10:07.808 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48205.log 2024-03-29T04:10:07.808 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52366.log.gz 2024-03-29T04:10:07.808 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44009.log 2024-03-29T04:10:07.808 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48205.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48463.log.gz 2024-03-29T04:10:07.808 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53468.log 2024-03-29T04:10:07.808 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44009.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48205.log.gz 2024-03-29T04:10:07.808 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48889.log 2024-03-29T04:10:07.808 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44009.log.gz 2024-03-29T04:10:07.809 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.53468.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47074.log 2024-03-29T04:10:07.809 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44268.log 2024-03-29T04:10:07.809 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53468.log.gz 2024-03-29T04:10:07.809 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47074.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48889.log.gz 2024-03-29T04:10:07.809 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48043.log 2024-03-29T04:10:07.809 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47074.log.gz 2024-03-29T04:10:07.809 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47620.log 2024-03-29T04:10:07.809 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51323.log 2024-03-29T04:10:07.810 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44268.log.gz 2024-03-29T04:10:07.810 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47620.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48043.log.gz 2024-03-29T04:10:07.810 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44942.log 2024-03-29T04:10:07.810 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47620.log.gz 2024-03-29T04:10:07.810 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46822.log 2024-03-29T04:10:07.810 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47452.log 2024-03-29T04:10:07.810 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46822.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44942.log.gz 2024-03-29T04:10:07.810 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44273.log 2024-03-29T04:10:07.811 INFO:teuthology.orchestra.run.smithi097.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51323.log.gz 2024-03-29T04:10:07.811 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46822.log.gz 2024-03-29T04:10:07.811 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44604.log 2024-03-29T04:10:07.811 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47452.log.gz 2024-03-29T04:10:07.811 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47242.log 2024-03-29T04:10:07.811 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43799.log 2024-03-29T04:10:07.811 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44273.log.gz 2024-03-29T04:10:07.811 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44604.log.gz 2024-03-29T04:10:07.811 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47746.log 2024-03-29T04:10:07.812 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47242.log.gz 2024-03-29T04:10:07.812 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43799.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-29T04:10:07.812 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47746.log: gzip -5 --verbose -- 89.5% /var/log/ceph/ceph-client.admin.51031.log 2024-03-29T04:10:07.812 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43799.log.gz 2024-03-29T04:10:07.812 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47746.log.gz 2024-03-29T04:10:07.812 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52618.log 2024-03-29T04:10:07.812 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45773.log 2024-03-29T04:10:07.812 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52618.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51031.log.gz 2024-03-29T04:10:07.813 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52324.log 2024-03-29T04:10:07.813 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52618.log.gz 2024-03-29T04:10:07.813 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51936.log 2024-03-29T04:10:07.813 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45773.log: /var/log/ceph/ceph-client.admin.52324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45185.log 2024-03-29T04:10:07.813 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-29T04:10:07.813 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4%/var/log/ceph/ceph-client.admin.51936.log: -- replaced with /var/log/ceph/ceph-client.admin.52324.log.gz 2024-03-29T04:10:07.813 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45479.log 2024-03-29T04:10:07.813 INFO:teuthology.orchestra.run.smithi097.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.45773.log.gz 2024-03-29T04:10:07.813 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45185.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51936.log.gz 2024-03-29T04:10:07.814 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52492.log 2024-03-29T04:10:07.814 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45185.log.gz 2024-03-29T04:10:07.814 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52282.log 2024-03-29T04:10:07.814 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45479.log.gz 2024-03-29T04:10:07.814 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50370.log 2024-03-29T04:10:07.814 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43715.log 2024-03-29T04:10:07.814 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52492.log.gz 2024-03-29T04:10:07.814 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50370.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52282.log.gz 2024-03-29T04:10:07.815 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45647.log 2024-03-29T04:10:07.815 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50370.log.gz 2024-03-29T04:10:07.815 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53510.log 2024-03-29T04:10:07.815 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45647.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43715.log.gz 2024-03-29T04:10:07.815 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52847.log 2024-03-29T04:10:07.815 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53510.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45647.log.gz 2024-03-29T04:10:07.815 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53553.log 2024-03-29T04:10:07.815 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53510.log.gz 2024-03-29T04:10:07.815 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49015.log 2024-03-29T04:10:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53332.log 2024-03-29T04:10:07.816 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52847.log.gz 2024-03-29T04:10:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49015.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53553.log.gz 2024-03-29T04:10:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45143.log 2024-03-29T04:10:07.816 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49015.log.gz 2024-03-29T04:10:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47200.log 2024-03-29T04:10:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45143.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.53332.log.gz --verbose 2024-03-29T04:10:07.816 INFO:teuthology.orchestra.run.smithi097.stderr: -- /var/log/ceph/ceph-client.admin.45311.log 2024-03-29T04:10:07.816 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45143.log.gz 2024-03-29T04:10:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50580.log 2024-03-29T04:10:07.817 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47200.log.gz 2024-03-29T04:10:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54034.log 2024-03-29T04:10:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50580.log: gzip -5 88.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.45311.log.gz /var/log/ceph/ceph-client.admin.49351.log 2024-03-29T04:10:07.817 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-29T04:10:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50105.log 2024-03-29T04:10:07.817 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50580.log.gz 2024-03-29T04:10:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49351.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54034.log.gz 2024-03-29T04:10:07.818 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44610.log 2024-03-29T04:10:07.818 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49351.log.gz 2024-03-29T04:10:07.818 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44945.log 2024-03-29T04:10:07.818 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44610.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50105.log.gz 2024-03-29T04:10:07.818 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52889.log 2024-03-29T04:10:07.818 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44610.log.gz 2024-03-29T04:10:07.818 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44945.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-29T04:10:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52889.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44945.log.gz 2024-03-29T04:10:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31165.log 2024-03-29T04:10:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.1.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52889.log.gz 2024-03-29T04:10:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48931.log 2024-03-29T04:10:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.tmp-client.admin.31165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52534.log 2024-03-29T04:10:07.819 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31165.log.gz 2024-03-29T04:10:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50913.log 2024-03-29T04:10:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52534.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48931.log.gz 2024-03-29T04:10:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45227.log 2024-03-29T04:10:07.820 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50913.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52534.log.gz 2024-03-29T04:10:07.820 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53739.log 2024-03-29T04:10:07.820 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50913.log.gz 2024-03-29T04:10:07.820 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47959.log 2024-03-29T04:10:07.820 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45227.log: /var/log/ceph/ceph-client.admin.53739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44264.log 2024-03-29T04:10:07.820 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45227.log.gz 2024-03-29T04:10:07.820 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47959.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53739.log.gz 2024-03-29T04:10:07.820 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50286.log 2024-03-29T04:10:07.821 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44275.log 2024-03-29T04:10:07.821 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44264.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47959.log.gz 2024-03-29T04:10:07.821 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50286.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44264.log.gz 2024-03-29T04:10:07.821 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51810.log 2024-03-29T04:10:07.821 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50286.log.gz 2024-03-29T04:10:07.821 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51239.log 2024-03-29T04:10:07.821 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44275.log: /var/log/ceph/ceph-client.admin.51810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44606.log 2024-03-29T04:10:07.821 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44275.log.gz 2024-03-29T04:10:07.821 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51239.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51810.log.gz 2024-03-29T04:10:07.822 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48763.log 2024-03-29T04:10:07.822 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46990.log 2024-03-29T04:10:07.822 INFO:teuthology.orchestra.run.smithi097.stderr: 90.5% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51239.log.gz -- replaced with /var/log/ceph/ceph-client.admin.44606.log.gz 2024-03-29T04:10:07.822 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-29T04:10:07.823 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53121.log 2024-03-29T04:10:07.823 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48085.log 2024-03-29T04:10:07.823 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48763.log.gz 2024-03-29T04:10:07.823 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48598.log 2024-03-29T04:10:07.823 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53121.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46990.log.gz 2024-03-29T04:10:07.823 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47158.log 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53121.log.gz 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48085.log.gz 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43525.log 2024-03-29T04:10:07.824 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48598.log: /var/log/ceph/ceph-client.admin.47158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49309.log 2024-03-29T04:10:07.825 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48598.log.gz 2024-03-29T04:10:07.825 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43525.log: gzip -5 87.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47158.log.gz 2024-03-29T04:10:07.825 INFO:teuthology.orchestra.run.smithi097.stderr: /var/log/ceph/ceph-client.0.46249.log 2024-03-29T04:10:07.825 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49309.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43525.log.gz 2024-03-29T04:10:07.825 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48331.log 2024-03-29T04:10:07.825 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49309.log.gz 2024-03-29T04:10:07.825 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44598.log 2024-03-29T04:10:07.825 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.46249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46249.log.gz 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52056.log 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44598.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48331.log.gz 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52140.log 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47494.log 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52056.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44598.log.gz 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43757.log 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52056.log.gz 2024-03-29T04:10:07.826 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52140.log.gz 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44807.log 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47494.log: /var/log/ceph/ceph-client.admin.43757.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47494.log.gz 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43757.log.gz 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51852.log 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44807.log.gz 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49723.log 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.51852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52576.log 2024-03-29T04:10:07.827 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51852.log.gz 2024-03-29T04:10:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49807.log 2024-03-29T04:10:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52931.log 2024-03-29T04:10:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52576.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49723.log.gz 2024-03-29T04:10:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49807.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52576.log.gz 2024-03-29T04:10:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44051.log 2024-03-29T04:10:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.52931.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52098.log 2024-03-29T04:10:07.828 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49807.log.gz 2024-03-29T04:10:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44051.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52931.log.gz 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37464.log 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48247.log 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52098.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44051.log.gz 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37464.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52098.log.gz 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50646.log 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37464.log.gz 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50955.log 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48247.log: /var/log/ceph/ceph-client.admin.50646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53697.log 2024-03-29T04:10:07.829 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48247.log.gz 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50955.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50646.log.gz 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51173.log 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47788.log 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53697.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50955.log.gz 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45353.log 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53697.log.gz 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43967.log 2024-03-29T04:10:07.830 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47788.log: 91.4%/var/log/ceph/ceph-client.admin.45353.log: -- replaced with /var/log/ceph/ceph-client.admin.51173.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45521.log 2024-03-29T04:10:07.831 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-29T04:10:07.831 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47788.log.gz 2024-03-29T04:10:07.831 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44608.log 2024-03-29T04:10:07.831 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45353.log.gz 2024-03-29T04:10:07.831 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45521.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43967.log.gz 2024-03-29T04:10:07.831 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49183.log 2024-03-29T04:10:07.831 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47032.log 2024-03-29T04:10:07.831 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44608.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45521.log.gz 2024-03-29T04:10:07.832 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43883.log 2024-03-29T04:10:07.832 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44608.log.gz 2024-03-29T04:10:07.832 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49183.log.gz 2024-03-29T04:10:07.832 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50017.log 2024-03-29T04:10:07.832 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47032.log: /var/log/ceph/ceph-client.admin.43883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45689.log 2024-03-29T04:10:07.832 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47032.log.gz 2024-03-29T04:10:07.832 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43883.log.gz 2024-03-29T04:10:07.832 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50017.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-29T04:10:07.833 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44271.log 2024-03-29T04:10:07.833 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45689.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50017.log.gz 2024-03-29T04:10:07.833 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52224.log 2024-03-29T04:10:07.833 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45689.log.gz 2024-03-29T04:10:07.833 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49057.log 2024-03-29T04:10:07.833 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44271.log: /var/log/ceph/ceph-client.admin.52224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50454.log 2024-03-29T04:10:07.833 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44271.log.gz 2024-03-29T04:10:07.833 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46948.log 2024-03-29T04:10:07.833 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52224.log.gz 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50156.log 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49057.log.gz 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53163.log 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50454.log.gz 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46948.log: /var/log/ceph/ceph-client.admin.50156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44946.log 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50198.log 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50156.log.gz 2024-03-29T04:10:07.834 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46948.log.gz 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49975.log 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53163.log.gz 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50198.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48678.log 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44946.log.gz 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50198.log.gz 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44943.log 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49975.log.gz 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53382.log 2024-03-29T04:10:07.835 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48678.log: gzip/var/log/ceph/ceph-client.admin.44943.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52702.log 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2%/var/log/ceph/ceph-client.admin.53382.log: -- replaced with /var/log/ceph/ceph-client.admin.48678.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44943.log.gz 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43569.log 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53382.log.gz 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47284.log 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52702.log: /var/log/ceph/ceph-client.admin.43569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53907.log 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52702.log.gz 2024-03-29T04:10:07.836 INFO:teuthology.orchestra.run.smithi097.stderr:gzip 88.9% -5 --verbose -- /var/log/ceph/ceph-client.admin.44944.log 2024-03-29T04:10:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47284.log: -- replaced with /var/log/ceph/ceph-client.admin.43569.log.gz 2024-03-29T04:10:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53907.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-29T04:10:07.837 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47284.log.gz 2024-03-29T04:10:07.837 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53907.log.gz 2024-03-29T04:10:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47368.log 2024-03-29T04:10:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44944.log: /var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53991.log 2024-03-29T04:10:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49225.log 2024-03-29T04:10:07.837 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44944.log.gz 2024-03-29T04:10:07.837 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53991.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43357.log 2024-03-29T04:10:07.838 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47368.log.gz 2024-03-29T04:10:07.838 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53991.log.gz 2024-03-29T04:10:07.838 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49933.log 2024-03-29T04:10:07.838 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49225.log.gz/var/log/ceph/ceph-client.admin.43357.log: 2024-03-29T04:10:07.838 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53247.log 2024-03-29T04:10:07.838 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2%/var/log/ceph/ceph-client.admin.49933.log: -- replaced with /var/log/ceph/ceph-client.admin.43357.log.gz 2024-03-29T04:10:07.838 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44603.log 2024-03-29T04:10:07.838 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49933.log.gz 2024-03-29T04:10:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48847.log 2024-03-29T04:10:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53247.log: /var/log/ceph/ceph-client.admin.44603.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-29T04:10:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48847.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53247.log.gz 2024-03-29T04:10:07.839 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44603.log.gz 2024-03-29T04:10:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48289.log 2024-03-29T04:10:07.839 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48847.log.gz 2024-03-29T04:10:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49267.log 2024-03-29T04:10:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.48289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44272.log 2024-03-29T04:10:07.839 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49267.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48289.log.gz 2024-03-29T04:10:07.840 INFO:teuthology.orchestra.run.smithi097.stderr: 84.4% -- replaced with /var/log/ceph/ceph.log.gz 88.3% 2024-03-29T04:10:07.840 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49267.log.gz 2024-03-29T04:10:07.840 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45563.log 2024-03-29T04:10:07.840 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-29T04:10:07.840 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44272.log: /var/log/ceph/ceph-client.admin.45563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45437.log 2024-03-29T04:10:07.840 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.2.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44272.log.gz 2024-03-29T04:10:07.840 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45563.log.gz 2024-03-29T04:10:07.840 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44947.log 2024-03-29T04:10:07.840 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37210.log 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45437.log: /var/log/ceph/ceph-client.admin.44947.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45437.log.gz 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44947.log.gz 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37210.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37210.log.gz 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43841.log 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47116.log 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.43841.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43841.log.gz 2024-03-29T04:10:07.841 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52182.log 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47116.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47116.log.gz 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45911.log 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52182.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52182.log.gz 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi097.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50538.log 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47704.log 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50538.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45911.log.gz 2024-03-29T04:10:07.842 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50538.log.gz 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50412.log 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47704.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47704.log.gz 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48805.log 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50834.log 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48805.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50412.log.gz 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48805.log.gz 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46688.log 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.50834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45269.log 2024-03-29T04:10:07.843 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50834.log.gz 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.2.46688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.46688.log.gz 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49141.log 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45269.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45269.log.gz 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46864.log 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47662.log 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46864.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49141.log.gz 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46864.log.gz 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53655.log 2024-03-29T04:10:07.844 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47662.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47662.log.gz 2024-03-29T04:10:07.845 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47326.log 2024-03-29T04:10:07.845 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53865.log 2024-03-29T04:10:07.845 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47326.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53655.log.gz 2024-03-29T04:10:07.845 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47326.log.gz 2024-03-29T04:10:07.845 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48973.log 2024-03-29T04:10:07.845 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45395.log 2024-03-29T04:10:07.845 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48973.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53865.log.gz 2024-03-29T04:10:07.845 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48973.log.gz 2024-03-29T04:10:07.846 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44611.log 2024-03-29T04:10:07.846 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45395.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45395.log.gz 2024-03-29T04:10:07.847 INFO:teuthology.orchestra.run.smithi154.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-29T04:10:07.848 INFO:teuthology.orchestra.run.smithi154.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-29T04:10:07.848 INFO:teuthology.orchestra.run.smithi154.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-29T04:10:07.853 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52660.log 2024-03-29T04:10:07.853 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49680.log 2024-03-29T04:10:07.853 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52660.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44611.log.gz 2024-03-29T04:10:07.853 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52660.log.gz 2024-03-29T04:10:07.853 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53949.log 2024-03-29T04:10:07.853 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49680.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49680.log.gz 2024-03-29T04:10:07.854 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53290.log 2024-03-29T04:10:07.854 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53949.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-29T04:10:07.854 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53290.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53949.log.gz 2024-03-29T04:10:07.854 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53290.log.gz 2024-03-29T04:10:07.854 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50328.log 2024-03-29T04:10:07.854 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48421.log 2024-03-29T04:10:07.854 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50328.log: 88.3% 92.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-29T04:10:07.854 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50328.log.gz 2024-03-29T04:10:07.855 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43483.log 2024-03-29T04:10:07.855 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49515.log 2024-03-29T04:10:07.855 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43483.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48421.log.gz 2024-03-29T04:10:07.855 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43483.log.gz 2024-03-29T04:10:07.855 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54074.log 2024-03-29T04:10:07.855 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53424.log 2024-03-29T04:10:07.855 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54074.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49515.log.gz 2024-03-29T04:10:07.855 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54074.log.gz 2024-03-29T04:10:07.855 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53424.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53424.log.gz 2024-03-29T04:10:07.861 INFO:teuthology.orchestra.run.smithi097.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-29T04:10:07.883 INFO:teuthology.orchestra.run.smithi097.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-29T04:10:07.891 INFO:teuthology.orchestra.run.smithi154.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-29T04:10:07.899 INFO:teuthology.orchestra.run.smithi154.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-29T04:10:07.914 INFO:teuthology.orchestra.run.smithi097.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-29T04:10:07.923 INFO:teuthology.orchestra.run.smithi154.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-29T04:10:07.925 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-03-29T04:10:07.925 INFO:teuthology.orchestra.run.smithi154.stderr:real 0m0.208s 2024-03-29T04:10:07.925 INFO:teuthology.orchestra.run.smithi154.stderr:user 0m0.830s 2024-03-29T04:10:07.925 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m0.063s 2024-03-29T04:10:07.926 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-29T04:10:07.961 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-29T04:10:08.046 INFO:teuthology.orchestra.run.smithi097.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-29T04:10:08.048 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-03-29T04:10:08.048 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m0.364s 2024-03-29T04:10:08.048 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m1.291s 2024-03-29T04:10:08.048 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m0.171s 2024-03-29T04:10:08.048 INFO:tasks.ceph:Archiving logs... 2024-03-29T04:10:08.049 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076/remote/smithi097/log 2024-03-29T04:10:08.050 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T04:10:08.449 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076/remote/smithi154/log 2024-03-29T04:10:08.450 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T04:10:08.597 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-29T04:10:08.628 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-29T04:10:08.629 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T04:10:08.631 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T04:10:08.709 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-29T04:10:08.720 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-29T04:10:08.721 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:10:08.724 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:10:08.735 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:10:08.737 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:10:08.739 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:10:08.739 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-03-29T04:10:08.739 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 64 +92us[ +101us] +/- 77ms 2024-03-29T04:10:08.740 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 -1268us[-1263us] +/- 69ms 2024-03-29T04:10:08.740 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv03.front.sepia.ceph.com 3 6 377 0 +1147us[+1152us] +/- 66ms 2024-03-29T04:10:08.740 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:10:08.741 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:10:08.741 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-03-29T04:10:08.742 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 2 +94us[ +88us] +/- 78ms 2024-03-29T04:10:08.742 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -1325us[-1325us] +/- 69ms 2024-03-29T04:10:08.742 INFO:teuthology.orchestra.run.smithi154.stdout:^* hv03.front.sepia.ceph.com 3 6 377 2 +1150us[+1144us] +/- 66ms 2024-03-29T04:10:08.742 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:10:08.743 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-29T04:10:08.755 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-29T04:10:08.755 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-29T04:10:08.770 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-29T04:10:08.852 DEBUG:teuthology.orchestra.run.smithi154:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-29T04:10:08.919 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:10:08.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:10:08.949 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:10:08.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:10:08.979 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-29T04:10:08.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-29T04:10:09.006 INFO:teuthology.task.internal:Duration was 587.123795 seconds 2024-03-29T04:10:09.007 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-29T04:10:09.023 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-29T04:10:09.024 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T04:10:09.027 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T04:10:09.067 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:10:09.070 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:10:09.375 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-29T04:10:09.375 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-03-29T04:10:09.375 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T04:10:09.434 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2024-03-29T04:10:09.435 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T04:10:09.455 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-29T04:10:09.455 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T04:10:09.476 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T04:10:09.523 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-29T04:10:09.523 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T04:10:09.545 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T04:10:09.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-29T04:10:09.674 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-29T04:10:09.674 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T04:10:09.701 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T04:10:09.728 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-29T04:10:09.739 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:10:09.743 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:10:09.766 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-03-29T04:10:09.794 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2024-03-29T04:10:09.810 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:10:09.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:10:09.836 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:10:09.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:10:09.864 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-29T04:10:09.875 INFO:teuthology.task.internal:Transferring archived files... 2024-03-29T04:10:09.876 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076/remote/smithi097 2024-03-29T04:10:09.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T04:10:09.912 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076/remote/smithi154 2024-03-29T04:10:09.912 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T04:10:09.948 INFO:teuthology.task.internal:Removing archive directory... 2024-03-29T04:10:09.948 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T04:10:09.951 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T04:10:09.998 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-29T04:10:10.012 INFO:teuthology.task.internal:Not uploading archives. 2024-03-29T04:10:10.013 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-29T04:10:10.028 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-29T04:10:10.028 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T04:10:10.031 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T04:10:10.043 INFO:teuthology.orchestra.run.smithi097.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 29 04:10 /home/ubuntu/cephtest 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi154.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 29 04:10 /home/ubuntu/cephtest 2024-03-29T04:10:10.053 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-29T04:10:10.065 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-29T04:10:10.089 INFO:teuthology.nuke:Checking targets against current locks 2024-03-29T04:10:10.115 DEBUG:teuthology.nuke:shortname: smithi097 2024-03-29T04:10:10.116 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T04:10:10.137 DEBUG:teuthology.nuke:shortname: smithi154 2024-03-29T04:10:10.138 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T04:10:10.141 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:51:28.156004', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIZEzUaPO6wk7oZ1TFA+ugiuBrfnoH0JnaSZXeUIPE1MO0Hym+6XkM68t6vlyVE4ughsiaFekDbZCWB3/gc3H/Y='} 2024-03-29T04:10:10.162 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629076', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:51:28.157853', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEWPhAtYviuzhJO7VkwnqL3RDKbYNrsCin0sYC7fKMpz9KQzEDIwmoNQsE1m1jOSCNOscybOnMO0aANEZJoPBJs='} 2024-03-29T04:10:10.185 INFO:teuthology.orchestra.console.smithi097:Power off 2024-03-29T04:10:10.186 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T04:10:10.210 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2024-03-29T04:10:10.210 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:10:10.231 INFO:teuthology.orchestra.console.smithi154:Power off 2024-03-29T04:10:10.231 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T04:10:10.244 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-29T04:10:10.254 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2024-03-29T04:10:10.255 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:10:10.279 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-03-29T04:10:14.245 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:10:14.280 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:10:14.355 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-03-29T04:10:14.390 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-03-29T04:10:18.358 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:10:18.391 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:10:18.470 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2024-03-29T04:10:18.506 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-03-29T04:10:18.572 INFO:teuthology.orchestra.console.smithi097:Power off completed 2024-03-29T04:10:22.509 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:10:22.622 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2024-03-29T04:10:22.723 INFO:teuthology.orchestra.console.smithi154:Power off completed 2024-03-29T04:10:22.828 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} duration: 587.1237950325012 failure_reason: 'Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining)' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=6cfc6d29007a4ef8880736ffa9fd1789 status: fail success: false 2024-03-29T04:10:22.829 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T04:10:22.909 INFO:teuthology.run:FAIL